Phase 6: Prompt Engineering & GenAI Feature Design · 40 min · Model playgrounds · v0 · Bolt.new
The Problem
A PM has an idea: an AI feature that turns a rambling voice memo into a structured meeting-action-item list. Following the course's process so far, the instinct is to go straight to a full PRD (Phase 4), complete with a cost model, an evaluation plan (Phase 5), and prompt specs (this phase's Lesson 1) — three to five days of work before a single real user has seen anything. Two weeks after that PRD is approved and engineering starts building, user interviews reveal the actual problem: users don't want structured action items extracted automatically, they want to edit the AI's interpretation before it's finalized, because the raw transcription is often ambiguous about who owns what. The core interaction model was wrong, and the team spent two weeks of engineering time building the wrong thing before finding out — a finding that a single afternoon with a model playground and five real user tests would have surfaced for near-zero cost.
This is the case for rapid prototyping as a distinct, earlier step: validate the riskiest assumption cheaply, before committing the cost of a full PRD and engineering build to it.
Choosing the right prototyping tool for the question
Different validation questions call for different tools, and picking the wrong one wastes the exact time rapid prototyping is supposed to save:
- Model playgrounds (Anthropic Console/Workbench, OpenAI Playground) are the right tool when the question is about the model's behavior itself — can it reliably do the core task at all, what does the output actually look like, how does it fail. This is the fastest, cheapest prototyping step and should almost always come first: there's no point building UI around a core capability you haven't confirmed works.
- AI-assisted UI builders (v0, Bolt.new, Replit) are the right tool when the question is about the interaction model — does a user actually want to interact with this capability the way you've imagined, is the proposed UX intuitive, does seeing a real (even rough) interface change how people react to the idea compared to describing it. These tools can produce a working, clickable interface in hours instead of days, letting a PM test the interaction model without waiting for an engineering sprint.
- A working interface is not always necessary. For some questions — is the core AI capability good enough, does a specific prompt approach handle a specific hard case — a Workbench session and a handful of pasted examples answers the question faster than any UI build.
Match the prototype's fidelity to the question, not to what's impressive
The most common rapid-prototyping trap is spending the "same day" budget polishing visual details of a UI builder output instead of testing the actual risky assumption. If the risky assumption is "can the model reliably extract action items from a rambling voice memo," a Workbench session testing that exact task against 10 real (or realistic) voice memo transcripts answers the question in an hour. If the risky assumption is "do users want to edit the AI's interpretation before it's finalized," a rough, even ugly, clickable v0 prototype that lets 5 real users try editing an example output answers that question far better than a polished but non-interactive mockup. Polish answers a different question (does this look professional) than validation does (was our core assumption right) -- spending scarce prototype time on the former instead of the latter is the trap.
Structuring a same-day prototype test
A prototype without a predefined hypothesis and kill criterion just produces a demo, not a validation result — and a demo, per Phase 5's opening lesson, is a sample size of one that tells you almost nothing. A structured test needs:
- A specific, falsifiable hypothesis — not "users will like this" (unfalsifiable) but "at least 4 of 5 test users will successfully identify and correct an AI misattribution in the action-item list without being prompted to look for one" (falsifiable, specific, testable in an afternoon).
- A kill criterion decided in advance — the result that means the core assumption was wrong and the idea needs to change materially or be shelved, decided before running the test, so a disappointing result can't be quietly reinterpreted as "close enough" under the pull of sunk cost.
- A minimal but real test population — even 5 real target users testing a rough prototype beats 50 internal team members testing a polished one, because internal team members already know the intended interaction model and can't validate whether it's discoverable to someone encountering it fresh.
A PM builds a v0 prototype of the voice-memo-to-action-items feature and tests it with 5 colleagues from the product team, who all rate it "very intuitive" and understand exactly how to use it. The PM concludes the interaction model is validated. What is the strongest critique of this test?
Testing with colleagues from the same product team introduces a severe bias: they've likely absorbed context about the intended interaction model just by being adjacent to the project, so their ability to use it intuitively says very little about whether a real target user encountering the feature cold would have the same experience. This is the same 'representative sample' principle from Phase 5's golden dataset lessons, applied to user testing instead of model evaluation -- a convenient, unrepresentative sample produces a result that looks reassuring but doesn't actually answer the validation question. The fix is testing with real target users, not simply more testers or more prototype polish.
From prototype to PRD: what a prototype can and can't tell you
A successful rapid prototype test is evidence the core idea and interaction model are worth the investment of a full PRD — it is not, by itself, sufficient to skip the PRD process. A prototype validates the riskiest early assumption cheaply; it does not validate cost at scale, does not establish the eval plan and quality bar (Phase 5), does not work through edge cases and failure UX (Phase 4), and does not assess responsible AI risk (Phase 7). Treating a well-received prototype as a finished spec skips exactly the work that turns a validated idea into a shippable, safe product.
A prototype test goes well: 5 target users all successfully used a rough v0 interface to correct AI-generated action items. The PM tells engineering "the prototype validates this feature, let's build it" and skips writing a full PRD, reasoning that the interaction model is already proven. What is the strongest critique of skipping the PRD?
A prototype and a PRD answer different questions, the same way offline eval and online eval do (Phase 5): the prototype cheaply validates whether the core idea and interaction model are worth pursuing, but says nothing about cost projections at scale, the evaluation plan and ship criteria, edge-case and failure-mode UX, or the responsible AI risk assessment that Phase 7 requires. A validated prototype is a strong reason TO write the PRD with confidence, not a reason to skip it -- the prototype result becomes an input to the PRD, not a replacement for it.