Phase 6: Prompt Engineering & GenAI Feature Design · 45 min · JSON schema · Tool/function calling · Structured outputs
The Problem
A PM specs an AI feature that extracts trip details (destination, dates, budget) from a free-text travel request and auto-fills a booking form. The first version asks the model to "return the destination, dates, and budget" in prose, and engineering writes a regex-based parser to pull the fields back out. It breaks constantly — the model writes "around $2,000" one time and "$2,000 (approx)" another, dates come back as "next Friday" without a resolved calendar date, and the parser silently drops fields it can't match. The PM assumed this was a model-quality problem. It's actually a product-architecture problem: free text was never going to reliably feed a form.
The fix is structured output: constraining the model to return data in a predefined, machine-parseable schema (typically JSON matching a JSON Schema definition) instead of free text a downstream system then has to guess how to parse.
Structured outputs: what changed and why it matters to product
Modern model APIs support schema-constrained generation — the caller supplies a JSON Schema (field names, types, required fields, enums for constrained choices), and the model's output is validated against that schema, often with the provider guaranteeing the output will parse as valid JSON matching the schema. This converts "the model wrote something resembling structured data most of the time" into "the model wrote data your code can json.loads() and trust the shape of, or the call fails explicitly instead of silently returning malformed prose."
The product implications are direct:
- Form-filling UX becomes reliable enough to auto-populate a real form (like the travel-booking example) instead of requiring a human to manually re-type what an AI already extracted — which is often the actual point of the feature.
- Anything feeding a downstream system (a database write, an API call, another model's input) needs structured output, because free text forces a fragile parsing step between the model and the system that actually depends on the data being correct.
- Constrained choice fields (enums) reduce a whole category of failure — asking the model to classify a support ticket into one of six defined categories via an enum field is far more reliable than asking it to "say what category this is" in prose and hoping the wording matches your six category names exactly.
Structured output narrows what can go wrong, it doesn't eliminate errors
A validated JSON schema guarantees shape (the right fields, the right types), not correctness (the right values). A trip-extraction feature can return a perfectly schema-valid JSON object with a hallucinated destination -- the schema validator will accept it happily, because "Springfield" is a syntactically valid string regardless of whether the user actually said Springfield. Structured output eliminates the parsing-failure category of bug and does nothing for the faithfulness category from Phase 5 -- both are still needed, doing different jobs.
Tool calling: the agent-calls-API pattern
Tool calling (also called function calling) extends structured output one step further: instead of just returning structured data, the model can decide when to invoke a predefined function/API, with what arguments, based on the conversation or task. The model doesn't execute the call itself — it emits a structured request ("call check_order_status with order_id: 48213"), your system executes the actual function, and the result is returned to the model to continue reasoning or respond to the user.
This is the mechanical foundation of every AI agent product: a customer support agent that can actually look up an order (not just describe how a user might look it up themselves), a coding assistant that can run a test, a scheduling assistant that can check calendar availability before proposing a time. The product design questions this raises — how much autonomy the agent has, whether it asks permission before acting — are the subject of Lesson 3; this lesson is about the mechanism that makes any of that possible in the first place.
A PM is designing a feature where an AI assistant should check a customer's order status when asked. The current implementation has the model generate a free-text response describing what the order status "probably is" based on general knowledge, with no actual system lookup. What is the correct fix, and why?
No amount of fine-tuning or model upgrading fixes the structural problem: the model has no access to the real order status and is generating a plausible-sounding guess, which is a hallucination by construction, not a capability gap. The correct fix is tool calling -- giving the model a defined function it can invoke to retrieve the actual order status from the real system, so its response is grounded in real data rather than invented. A disclaimer doesn't fix the underlying issue; it just warns the user about a problem the architecture should eliminate.
JSON schema reliability and failure handling
Even with provider-guaranteed schema validity, a PM should still specify the product's behavior for the cases where the content is wrong even though the shape is valid, and for the rarer case where generation fails outright:
- Schema-valid but wrong value (the hallucinated-destination case above) — this is caught by the eval and faithfulness practices from Phase 5, not by schema validation. The PRD should specify what happens when a downstream automation acts on a value that later turns out wrong: is there a confirmation step before an auto-filled form submits, or does a low-confidence field get flagged for human review instead of silently auto-filled?
- Generation failure (the call errors, times out, or a provider-side issue prevents a valid response) — the product needs an explicit fallback: retry with backoff, degrade to a simpler prompt, or surface a clear "couldn't extract this automatically, please fill in manually" state. Silently failing to a broken or empty form is a worse experience than the manual-entry flow it was meant to replace.
- Tool call failure (the underlying function/API errors) — same principle: the agent needs a defined behavior for a failed tool call, not an assumption that tool calls always succeed. Does it retry, ask the user for another approach, or escalate to a human?
Reading the signal in an interview
When a candidate is asked to design a feature that fills a form or takes an action from natural language, interviewers are listening for whether structured outputs and tool calling come up unprompted, and whether the candidate specifies failure behavior without being asked. A candidate who says "the AI extracts the info and fills the form" without addressing what happens when extraction is wrong or fails has described the happy path only — exactly the same gap Phase 4 flagged for designing around failure in a PRD, now showing up at the prompt/architecture level.
A candidate designs a resume-parsing feature: "The AI reads the resume and extracts name, email, and work history into structured fields that auto-populate the applicant's profile." An interviewer asks what happens if the AI extracts an incorrect email address. The candidate says "that would be rare with a good model." What is the strongest critique of this answer?
"That would be rare" answers a question about probability when the interviewer asked a question about design: what happens when it does occur, however rarely. At any real product scale, a low error rate still produces a nonzero number of affected users, and an incorrect auto-filled email (used for account communication, verification, etc.) is a high-consequence field to get silently wrong. The strong answer specifies a mitigation independent of the error rate -- for example, flagging low-confidence or newly-extracted contact fields for a one-tap user confirmation before they're treated as final -- rather than betting the whole design on the error simply not happening.