Phase 0: AI Product Management Foundations · 45 min · AI Product Lifecycle Map · Python
The Problem
A PM at a mid-size fintech spends three weeks on a PRD for an AI-powered spending-insights assistant. It's genuinely excellent: clear user stories, a tight scope, explicit non-goals, a cost budget per query, even a section on tone and disclaimers. Engineering builds against it almost without friction. The demo two weeks later is great. Everyone claps.
Six weeks after launch, the feature is quietly disabled. Not because the PRD was wrong — because nobody on the team had defined what "correct" meant for a non-deterministic output before it shipped, so there was no eval suite to catch it drifting into confidently wrong answers about a user's own transactions. And nobody had wired up production monitoring, so the team's first signal that anything was broken was a spike in support tickets and a churn-risk flag from a large customer, three weeks after the regression started. The PRD stage was flawless. The eval stage and the monitor stage didn't exist. The feature failed anyway.
This is the single most common failure mode in AI product work, and it's a lifecycle failure, not a document failure. Traditional software PMs can often get away with treating stages as a checklist — write the spec, ship it, move on, and if something breaks, the bug tracker catches it because bugs are usually deterministic and reproducible. AI features don't fail that way. They degrade statistically, on inputs nobody tested, in ways a bug tracker doesn't catch and a well-written PRD doesn't prevent. The only defense is treating opportunity, data, PRD, eval, build, responsible AI review, launch, and monitoring as one connected system where a gap at any stage becomes a failure at a later, more expensive stage.
This lesson is the map. Every later phase of this course builds deep skill in one stage of it — this lesson previews all of them so you know where you are, and where you're weak, at any point in a real project.
The eight stages
| Stage | Course phase | What a PM actually produces |
|---|
| Opportunity | Phase 2 — Discovery & AI Product Strategy | An opportunity assessment: build/buy/fine-tune call, feasibility and risk triage, a roadmap slot (Project 1) |
| Data | Phase 3 — Data Strategy for AI Products | A data audit, a labeling/annotation plan, a feedback-loop design, a privacy/governance review |
| PRD | Phase 4 — Writing AI Product Requirements | An AI PRD: scoped behavior spec, prompt spec, explicit non-deterministic-behavior bounds, failure-mode design, cost/latency budget (Project 2) |
| Eval | Phase 5 — Evaluation & Quality for AI Products | A rubric, an eval set, an LLM-as-judge or human eval pipeline, offline results, a ship/no-ship recommendation (Project 3) |
| Prompt / Build | Phase 6 — Prompt Engineering & GenAI Feature Design | Working prompts, structured outputs or tool-calling specs, a prototype validated against the eval set |
| Responsible AI review | Phase 7 — Responsible AI, Governance & Risk | A bias/fairness check, a transparency and disclosure plan, an incident-response plan (Project 4) |
| Launch | Phase 8 — Launch, Metrics & Scaling AI Products | A rollout plan (canary/staged), defined AI-specific KPIs, a go/no-go gate |
| Monitor | Phase 8 — Launch, Metrics & Scaling AI Products | Drift dashboards, unit-economics tracking, a retraining/re-prompting trigger |
Two things worth noticing in that table. First, Opportunity and Data come before you write a word of PRD — a PRD written before you know whether the data exists to support the feature is fiction with formatting. Second, Launch and Monitor are the same phase of this course on purpose: in AI products, launch isn't an event, it's the start of a measurement period. A "launched" feature with no monitoring isn't actually done — it's an unmonitored bet.
Every stage produces evidence, not just a document
It's tempting to read "PRD stage" as "write the PRD document" and consider the stage complete. The actual definition of done at each stage is evidence that a specific question was answered — not that a document with the right section headers exists. A PRD is done when it specifies what the model should do when it's uncertain, not just what it should do when it works. An eval stage is done when you have a scored result on a held-out set, not when you have a rubric sitting in a doc nobody ran. Treat every artifact as a proxy for a question — if you can't say what question it answers, it isn't evidence, it's paperwork.
Why the order (mostly) matters
The lifecycle isn't perfectly linear — you'll loop back from Eval to PRD when a rubric reveals the spec was underspecified, and you'll loop back from Monitor to Data when drift traces to a shifting input distribution. But the dependencies are real and mostly one-directional:
- You can't write a trustworthy PRD without knowing what data you actually have (Data before PRD).
- You can't run a meaningful eval without a PRD that defines correct and incorrect behavior specifically enough to grade against (PRD before Eval).
- You can't respectably build or prompt-engineer toward a target you haven't defined a passing score for (Eval before Build) — otherwise "looks good in the demo" quietly becomes your eval.
- You can't do a responsible AI review of a system that doesn't exist yet in a testable form (Build before Review).
- You can't monitor what you didn't instrument before launch (Launch and Monitor are built together, not sequentially).
Skip a stage, and the work doesn't disappear — it just resurfaces later, more expensive, and usually as an incident instead of a checklist item.
An AI resume-screening feature launches on schedule with a polished PRD and a clean UI. Three months later, a journalist reports the tool systematically down-ranks candidates from a specific university whose name correlates with a protected demographic. The team scrambles to add a fairness check post-incident. Which lifecycle stage was actually skipped, and which stage merely surfaced the failure?
This is a Responsible AI Review gap (Phase 7). A disparate-impact bias audit is exactly the artifact designed to catch a demographic-correlated ranking pattern before it reaches users — that's the specific evidence this stage produces. The press report is the symptom, appearing at what looks like the Launch/Monitor stage, but the actual missing work happened two stages earlier. Rewriting the PRD or collecting more data doesn't address a fairness problem that a structured bias audit is built to catch; only doing the skipped stage does.
Using the map to diagnose, not just plan
The most practical use of this lifecycle isn't planning a new feature — it's diagnosing a stuck or failing one. When a project stalls, "what stage are we actually in" is usually a more useful question than "what's the next task." A team that says "we're basically done, just polishing the PRD" while they have no eval set and no rubric isn't in the PRD stage — they're stuck before Eval and don't know it yet, because nothing forced them to notice.
This diagnostic habit is why the course phases exist in this order: Phase 2 (Opportunity) through Phase 8 (Launch/Monitor) each hand you a specific tool for one stage, in the sequence a real project needs them. By the time you reach Phase 9's capstone, you'll assemble Projects 1 through 4 — the opportunity assessment, the PRD, the eval plan, and the responsible AI review — into one case study that demonstrates you can operate the whole lifecycle, not just the parts that make for a good demo.
A team has a working prototype that performs well on every prompt the team personally tried (the "Build" stage feels done), and they're eager to move straight to Launch. What stage almost certainly needs to happen first, and why does "it worked when I tried it" not substitute for it?
This is an Eval gap. "It worked when I tried it" is anecdote, not evidence — it reflects whatever prompts happened to be top of mind for one person, not the distribution of real user inputs, edge cases, and adversarial phrasing the system will actually face. The Eval stage (Phase 5) exists precisely to replace that anecdote with a scored result on a held-out, representative set, so "done" means something a stranger could verify. Skipping straight to Launch means the first real eval run happens in production, on real users, which is the most expensive place to discover a failure mode.