Phase 4: Writing AI Product Requirements · 45 min · Markdown · Python
Cost & Latency Budgeting in the PRD
A feature that's profitable at 1,000 users and bankrupts the unit economics at 100,000 wasn't actually scoped — it was guessed at.
Hiring signal: The build plan's research is specific: a real AI PRD structure includes 'a cost projection table (queries/day x tokens/query x $/month at launch, growth, and scale)' as a named, load-bearing section — Reforge's generative-AI PRD guide treats this as non-negotiable. A PM who can build and defend this table in an interview is demonstrating exactly the token-economics fluency that separates a candidate who can 'hold a real technical conversation with ML engineers' from one who can only talk about the feature conceptually.
What you will learn
- Explain token economics: input/output pricing and how context window size drives cost
- Build a unit cost model per request, per user, and per feature
- Set a defensible latency SLA for a generative feature and explain the user-experience tradeoffs behind it
- Produce a 3-stage (launch/growth/scale) cost projection table for a real AI feature
The Problem
A PM ships an AI-powered document-analysis feature bundled into a $29/month subscription tier at no usage limit, reasoning that "AI is cheap now" and the feature will drive upgrades. The feature retrieves relevant context from a user's uploaded documents and sends it, along with the user's question, to a foundation-model API on every query. At launch, with 2,000 beta users making a few queries a week, the AI spend is a rounding error against revenue. Eight months later, at 80,000 subscribers with power users running dozens of queries a day against increasingly large document sets, the AI API bill is quietly exceeding the entire tier's subscription revenue — the feature that was supposed to drive upgrades is now actively losing money on every heavy user, and nobody sized this at launch because nobody built a cost model past the beta cohort's usage pattern.
This is a completely avoidable failure. Unlike model quality or user adoption, the cost of a generative feature at any given usage level is a knowable, computable number before launch — token pricing is public, and query volume can be estimated with the same rigor a traditional PM would apply to any other capacity-planning problem. The fact that many AI PRDs skip this isn't because it's hard to model; it's because "AI is cheap" is treated as a vibe instead of the specific, checkable claim it actually is.
Token Economics: The Building Blocks
Foundation-model APIs price by token — a token is roughly 3/4 of a word in English — with separate rates for input tokens and output tokens, and output tokens are typically priced several times higher than input tokens because generation is more computationally expensive than reading context. This asymmetry matters directly for product design: a feature that stuffs a large document into the context window (many cheap input tokens) but returns a short answer (few expensive output tokens) has a very different cost profile than a feature that takes a short prompt but generates a long report (few cheap input tokens, many expensive output tokens). A PM sizing cost needs to estimate both halves separately, not a single blended "cost per query" number pulled from a rough guess — the ratio of input to output tokens for a specific feature is usually the single biggest lever on unit cost, bigger than which model tier gets chosen.
Context window size is a recurring cost, not a one-time design choice
Every additional document, every turn of conversation history, every piece of retrieved context added to a prompt gets re-sent and re-billed on every single query in that session, not once. A RAG feature that retrieves 5,000 tokens of context per query costs roughly 5,000 input tokens every query, and if a design decision doubles the retrieval window "to improve quality," that's not a one-time engineering tradeoff — it's a permanent doubling of the input-token cost of every future query, at every future usage level. Cost modeling has to happen at the design-decision level (how much context does this feature send, every time), not just at the final feature level.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Building the Unit Cost Model, Latency SLAs for Generative Features, The 3-Stage Cost Projection Table, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy