Phase 4: Writing AI Product Requirements · 40 min · Product Compass AI PRD template · Markdown
The AI PRD
If your problem statement says 'AI', you've written a solution statement wearing a problem statement's clothes.
What you will learn
- Explain why an AI PRD's problem statement should be written without ever using the word 'AI'
- Structure an AI PRD's core sections: problem statement, success criteria, leading indicators, and the fine-tune-vs-prompt decision
- Distinguish a leading indicator (predicts future success) from a lagging indicator (confirms past success) in an AI feature's metrics section
- Generate a complete AI PRD skeleton for a real feature idea using a structured template
The Problem
A PM at a mid-size e-commerce company drafts a PRD titled "AI-Powered Product Description Generator." The opening paragraph reads: "We will use a large language model to automatically generate product descriptions, reducing manual copywriting effort." Engineering builds it. Three months later, adoption is near zero. When the PM finally talks to the merchandising team who was supposed to use it, she learns the actual problem: merchandisers spend six hours a week writing descriptions for a long tail of low-volume SKUs that never get written at all, so those products sit live with a blank description field and never sell. The AI generator she shipped writes fine descriptions — for products merchandisers were already going to write descriptions for anyway, because those are the ones anyone remembered to open a ticket for. It solved a technology story, not the actual problem.
This is the single most common failure mode in AI PRDs, and it has a specific, teachable fix: the problem statement should never mention AI. Miqdad Jaffer, a PM at OpenAI, formalized this as the first rule in his widely-used AI PRD template — if you can't state the user's problem without referencing the technology you plan to use to solve it, you haven't found the problem yet, you've found a solution and back-filled a justification. "Merchandisers can't keep up with description writing for long-tail SKUs, so 40% of those products go live with no description and a 60% lower conversion rate" is a problem statement. "We need an AI description generator" is not — it's an implementation detail masquerading as a business case, and it will not survive contact with an actual user's actual workflow the way this lesson's opening example didn't.
The AI PRD's structure
A traditional PRD and an AI PRD share a skeleton — problem, users, success metrics, scope, requirements — but an AI PRD needs sections a deterministic feature never requires, because the system's behavior is a distribution, not a fixed output. The structure this course uses, synthesized from Jaffer's Product Compass template and Reforge's generative-AI PRD guide, has seven load-bearing sections:
- Problem statement — written with zero mentions of AI, ML, or the specific model. States the user's problem and its cost in a business metric.
- Success criteria and leading indicators — the lagging business metric the feature is meant to move, plus the leading indicators that will show progress toward it long before the lagging metric has enough data to move.
- Fine-tune vs. prompt decision — an explicit, justified call on whether the feature needs a fine-tuned model, a prompted foundation model, or a RAG pipeline, with the tradeoff reasoning written down, not left implicit in an engineering Slack thread.
- Non-deterministic acceptance criteria — covered in depth in Lesson 2, but scoped here: what "correct" means for a system that won't produce the same output twice.
- Data provenance and freshness — where the system's inputs come from, how current they need to be, and what happens when they're stale.
- Cost projection table — covered in depth in Lesson 5: queries/day × tokens/query × $/month at launch, growth, and scale.
- Guardrails and feedback loops — what the system must never do, and how usage data gets back into improving it.
The rest of this lesson focuses on sections 1 and 2, because they're the ones that determine whether the other five sections end up solving the right problem at all.
The "no AI in the problem statement" rule has a simple test
Delete every occurrence of "AI," "ML," "LLM," "model," and the specific product name from your problem statement's first paragraph. If what's left still clearly states a user's problem and its cost, you passed. If what's left is vague, circular, or doesn't make sense without the technology reference, you wrote a solution statement. This test takes thirty seconds and catches the majority of first-draft AI PRDs, including ones written by experienced traditional PMs new to AI features — the instinct to lead with the exciting technology is strong and needs to be actively suppressed.
Success criteria: leading indicators, not just lagging ones
A lagging indicator confirms the feature worked after the fact: revenue lift, support-ticket deflection rate, retention delta. These are the metrics that go in an exec readout, and they matter — but they're slow. A generative feature that ships today might not move retention in a statistically meaningful way for six to eight weeks, and by the time that number is in, you've either shipped three more iterations blind or frozen the roadmap waiting for a number that was always going to take that long.
A leading indicator is something you can observe within days of launch that's causally connected to the lagging metric and gives you an early read on whether you're on the right track. For the product-description example: a leading indicator might be "percentage of generated descriptions merchandisers publish without editing" (a proxy for output quality) or "median time from SKU creation to a live description" (a proxy for whether the actual workflow gap is closing). Both are observable in the first week of a beta rollout. Neither is the real business metric — description quality doesn't equal revenue — but both are leading, causally connected signals that let a PM course-correct before the lagging metric has enough sample size to say anything.
The mistake to avoid is picking a leading indicator that's easy to move but not actually connected to the outcome — "number of times the generate button was clicked" moves the moment you ship a button, regardless of whether the feature helps anyone. A leading indicator needs the same scrutiny as a lagging one: would moving this number, on its own, give you real evidence the underlying problem is getting solved?
A PM's success-criteria section for an AI-powered support-ticket triage feature lists exactly one metric: "reduce average ticket resolution time by 20% within 6 months." The feature ships. Three weeks in, a stakeholder asks how it's going, and the PM has no answer beyond "we'll know at the 6-month mark." What's the specific gap in this success-criteria section, and what should have been added?
A single lagging metric with a long time horizon is a common and correctable gap. The fix isn't lowering the target or changing document format -- it's adding leading indicators the team can observe within days or weeks: e.g., "percentage of tickets the triage model routes to the correct queue on first pass" or "percentage of triage suggestions a human agent overrides." Both are observable almost immediately and causally connected to resolution time, so they let the PM see early whether the feature is working before three, four, or five months pass with zero signal. A PRD success-criteria section without at least one leading indicator leaves the team flying blind for most of the measurement window.
The fine-tune-vs-prompt decision belongs in the PRD
Traditional PRDs rarely specify implementation approach — that's usually left to engineering. An AI PRD is different: whether a feature is built with a prompted foundation model, a fine-tuned model, or a RAG pipeline over proprietary data has direct product consequences — cost per query, latency, how fast the feature can incorporate new information, and how much control the team has over failure modes. A PM who writes "the model will be fine-tuned on our support transcripts" without justifying why prompting plus retrieval wasn't sufficient is making an expensive, slow-to-reverse decision without writing down the reasoning — and six months later, when someone asks why the team took on a fine-tuning pipeline's maintenance burden, there's no record of what alternative was considered or rejected and why.
A PRD for a legal-contract clause classifier proposes fine-tuning a model on 50,000 labeled historical clauses. The classification taxonomy (12 clause types) has been stable for 8 years and changes maybe once every few years. A reviewer asks whether prompting plus retrieval would work instead. What's the strongest argument for fine-tuning being the right call here, specifically?
The fine-tune-vs-prompt decision hinges on exactly this kind of context, not a universal rule. In the support-triage example, a quarterly-changing taxonomy made a fine-tuned classifier a maintenance burden that couldn't keep pace, so a prompted approach won. Here, an 8-year-stable, rarely-changing taxonomy removes that specific downside of fine-tuning — the model won't need frequent retraining — while potentially gaining the classification consistency and lower per-query cost a fine-tuned model can offer at high volume. The PRD should still name this reasoning explicitly, the same way the support-triage example did, rather than defaulting to either approach as a rule of thumb.
Build
Build it
The code artifact lesson_04_01_the_ai_prd.py implements an AI PRD skeleton generator:
SuccessMetric — a dataclass distinguishing LEADING from LAGGING indicators, each with a target, a measurement window, and the causal link it's meant to serve as evidence forPRDSection enum covering the seven load-bearing sections described aboveAIPRDSkeleton — a dataclass holding the problem statement, an explicit "AI-mention check" flag, a list of SuccessMetric objects, and the fine-tune-vs-prompt decision with its justificationcheck_problem_statement_for_ai_mentions() — a simple text scan that flags AI/ML/LLM/model references in the problem statement, mirroring the thirty-second test from this lesson's calloutrender_prd_skeleton() — produces a markdown PRD skeleton with all seven sections populated- A worked example: an AI PRD skeleton for a customer-support ticket triage feature at a mid-size SaaS company, with a problem statement that passes the AI-mention check and a mix of leading and lagging success metrics
"""
lesson_04_01_the_ai_prd.py
An AI PRD skeleton generator: takes structured inputs (problem statement,
success metrics, the fine-tune-vs-prompt decision) and renders a markdown
PRD skeleton following the seven-section structure from Product Compass's
AI PRD template and Reforge's generative-AI PRD guide.
Also implements the "no AI in the problem statement" check as a runnable
function, not just a lesson exercise -- this is the artifact a PM would
actually keep and reuse across every AI feature they scope.
References:
- Product Compass AI PRD Template: https://www.productcompass.pm/p/ai-prd-template
- Aakash Gupta, How to Write an AI PRD: https://www.news.aakashg.com/p/ai-prd
Usage:
python lesson_04_01_the_ai_prd.py
No external dependencies, no network calls.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from enum import Enum
class MetricType(Enum):
LEADING = "leading"
LAGGING = "lagging"
@dataclass
class SuccessMetric:
name: str
metric_type: MetricType
target: str
measurement_window: str
causal_link: str # why this metric is believed to predict/confirm the outcome
AI_MENTION_PATTERN = re.compile(
r"\b(ai|artificial intelligence|ml|machine learning|llm|large language model|"
r"model|gpt|claude|gemini|chatbot|generative)\b",
re.IGNORECASE,
)
def check_problem_statement_for_ai_mentions(problem_statement: str) -> list[str]:
"""Returns every AI/ML/LLM/model-adjacent term found in the problem
statement. An empty list means the statement passes the 'no AI
mentioned' test from the lesson's callout."""
return sorted(set(m.group(0).lower() for m in AI_MENTION_PATTERN.finditer(problem_statement)))
@dataclass
class FineTuneVsPromptDecision:
choice: str # "prompted foundation model" | "fine-tuned model" | "RAG pipeline"
justification: str
alternatives_considered: list[str] = field(default_factory=list)
@dataclass
class AIPRDSkeleton:
title: str
problem_statement: str
affected_users: str
problem_cost_metric: str
success_metrics: list[SuccessMetric]
fine_tune_vs_prompt: FineTuneVsPromptDecision
data_provenance: str
data_freshness_requirement: str
guardrails: list[str]
feedback_loop: str
def ai_mention_check(self) -> list[str]:
return check_problem_statement_for_ai_mentions(self.problem_statement)
def render_prd_skeleton(prd: AIPRDSkeleton) -> str:
mentions = prd.ai_mention_check()
check_line = (
"PASS -- no AI/ML/LLM/model terms found in the problem statement"
if not mentions
else f"FAIL -- found: {', '.join(mentions)} (rewrite before proceeding)"
)
lines = [
f"# {prd.title}",
"",
"## 1. Problem Statement",
"",
prd.problem_statement,
"",
f"**Affected users:** {prd.affected_users}",
f"**Cost of the problem:** {prd.problem_cost_metric}",
f"**AI-mention check:** {check_line}",
"",
"## 2. Success Criteria",
"",
"| Metric | Type | Target | Window | Why it's evidence |",
"|---|---|---|---|---|",
]
for m in prd.success_metrics:
lines.append(
f"| {m.name} | {m.metric_type.value} | {m.target} | "
f"{m.measurement_window} | {m.causal_link} |"
)
lines += [
"",
"## 3. Fine-Tune vs. Prompt Decision",
"",
f"**Chosen approach:** {prd.fine_tune_vs_prompt.choice}",
f"**Justification:** {prd.fine_tune_vs_prompt.justification}",
"**Alternatives considered:**",
]
for alt in prd.fine_tune_vs_prompt.alternatives_considered:
lines.append(f"- {alt}")
lines += [
"",
"## 4. Data Provenance & Freshness",
"",
f"**Provenance:** {prd.data_provenance}",
f"**Freshness requirement:** {prd.data_freshness_requirement}",
"",
"## 5. Guardrails",
"",
]
for g in prd.guardrails:
lines.append(f"- {g}")
lines += [
"",
"## 6. Feedback Loop",
"",
prd.feedback_loop,
]
return "\n".join(lines)
# ── Worked example ───────────────────────────────────────────────────────────
# A support-ticket triage feature at a mid-size SaaS company. The problem
# statement is written to pass the AI-mention check; success metrics mix
# leading and lagging indicators deliberately.
def build_example_prd() -> AIPRDSkeleton:
return AIPRDSkeleton(
title="Support Ticket Triage & Routing",
problem_statement=(
"New support tickets sit in a shared inbox for a median of 4.2 hours "
"before a human agent reads them, categorizes them, and routes them "
"to the right team (billing, technical, account management). During "
"that window, 18% of tickets are mis-routed on the first attempt, "
"adding a further 6+ hour delay while the receiving team bounces "
"them back. Customers on the paid support tier are contractually "
"owed a 2-hour first-response time, and the team misses that SLA "
"on roughly 1 in 4 tickets during the mis-route path."
),
affected_users=(
"Support agents triaging the shared inbox, and paid-tier customers "
"whose tickets miss the 2-hour first-response SLA."
),
problem_cost_metric="SLA miss rate on paid tier: 24% of tickets; average agent time spent on manual routing: 45 min/day",
success_metrics=[
SuccessMetric(
name="First-pass routing accuracy",
metric_type=MetricType.LEADING,
target=">=90%",
measurement_window="First 2 weeks post-launch",
causal_link="Directly reduces the mis-route path that causes most SLA misses",
),
SuccessMetric(
name="Agent override rate",
metric_type=MetricType.LEADING,
target="<15%",
measurement_window="First 2 weeks post-launch",
causal_link="A high override rate signals agents don't trust the routing, predicting low adoption before the SLA metric would show it",
),
SuccessMetric(
name="Paid-tier 2-hour SLA compliance",
metric_type=MetricType.LAGGING,
target="95% (up from 76%)",
measurement_window="8 weeks post-launch",
causal_link="The actual business outcome this feature exists to move",
),
SuccessMetric(
name="Manual routing time per agent",
metric_type=MetricType.LAGGING,
target="<10 min/day (down from 45 min/day)",
measurement_window="8 weeks post-launch",
causal_link="Confirms the workflow-time savings the business case was built on",
),
],
fine_tune_vs_prompt=FineTuneVsPromptDecision(
choice="Prompted foundation model with a routing taxonomy in the system prompt, plus retrieval over the last 90 days of resolved tickets",
justification=(
"The routing taxonomy changes quarterly as new product lines "
"ship, so a fine-tuned classifier would need retraining on the "
"same cadence. A prompted approach with retrieval over recent "
"resolved tickets adapts to taxonomy changes by updating the "
"prompt and the retrieval corpus, with no retraining pipeline "
"to maintain."
),
alternatives_considered=[
"Fine-tuned classifier on 90 days of historical tickets -- rejected: retraining cadence can't keep pace with quarterly taxonomy changes",
"Simple keyword-rule router -- rejected: piloted for 2 weeks, 61% accuracy, well below the 90% target",
],
),
data_provenance="Retrieval corpus is the last 90 days of resolved, human-labeled tickets from the existing help desk system (Zendesk export); no external or purchased data.",
data_freshness_requirement="Retrieval corpus refreshed nightly; taxonomy prompt updated within 48 hours of any new product line launch.",
guardrails=[
"Never auto-close or auto-resolve a ticket -- routing only, a human always handles the actual response",
"Any ticket containing legal, security, or safety keywords is always routed to a human triage queue regardless of model confidence",
"If routing confidence is below 60%, the ticket is flagged 'needs manual review' rather than routed silently",
],
feedback_loop=(
"Every agent override (routing the ticket differently than the model "
"suggested) is logged with the original suggestion, the agent's "
"correction, and a free-text reason code. Weekly, the top override "
"patterns are reviewed and fed back into the routing taxonomy prompt."
),
)
if __name__ == "__main__":
prd = build_example_prd()
print(render_prd_skeleton(prd))
print()
mentions = prd.ai_mention_check()
print(
"Takeaway: the problem statement "
+ ("passes" if not mentions else "FAILS")
+ " the AI-mention check, and the success-criteria table mixes "
"leading indicators (routing accuracy, override rate) with lagging "
"ones (SLA compliance, agent time saved) -- exactly the mix this "
"lesson argues every AI PRD needs."
)
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Check Yourself — plus a hands-on lab, quiz, and project artifact.
Create a free account — the opening phases of 29 of 35 courses are free, no credit card.
Browse all courses · View pricing · DeVenture Academy