Phase 5: Evaluation & Quality for AI Products · 140 min · Golden datasets · Eval rubrics · DeepEval
The Problem
You've now worked through the full arc of an eval system: why probabilistic products need one at all (Lesson 1), how to design a human rubric that measures something real (Lesson 2), how automated pipelines extend that judgment to scale (Lesson 3), how offline and online testing answer different questions (Lesson 4), how to interpret hallucination, bias, and safety metrics without computing them (Lesson 5), and how to turn a dashboard full of mixed signals into a real ship/no-ship call (Lesson 6). Each of those lessons taught one piece in isolation. A real AI PM has to assemble all six pieces into a single artifact before a feature ever reaches a data science team for execution — because "we'll figure out evaluation once it's built" is exactly the eval-debt failure mode from Lesson 1, just deferred to a more expensive moment.
This project is that artifact. You will produce a complete evaluation plan, a scoring rubric, and a sample scoring pass for the same AI feature you carried through Phase 2's opportunity assessment and Phase 4's PRD. If you haven't been carrying a single feature through the course, pick one now — a specific, real or realistic AI product feature with enough surface area to need genuine evaluation (a support-ticket summarizer, a resume screener, a shopping assistant, a meeting-notes generator, or similar). This deliverable becomes the direct input to Phase 7's responsible AI review, and eventually one full section of the Phase 9 capstone case study — so build it as if a real data science team is going to receive it and start executing against it next week, because in the course's continuity, that's exactly what happens next.
What a complete evaluation plan contains
A data science team receiving a PM-authored evaluation plan should be able to start building against it without a clarifying-questions meeting. That means the plan has to specify, concretely, not vaguely:
- Golden dataset design (Lesson 1): what categories of input it must cover (happy path, edge case, at least one known or anticipated failure mode, adversarial if relevant), a target size, and who owns adding new examples over time as production surfaces new cases.
- Human eval rubric (Lesson 2): 3-5 weighted, anchored criteria specific to this feature — not generic "quality" — plus an explicit pass/fail rule for any catastrophic failure type, and a sampling strategy for ongoing human review once the feature is live.
- Automated metrics (Lesson 3): which DeepEval/Confident-AI-style metrics apply (faithfulness, relevance, bias, toxicity, or others specific to the feature), what threshold constitutes a pass, and a calibration cadence — how often the automated judge gets checked against human scores.
- Offline/online testing strategy (Lesson 4): what the golden-set eval delta process looks like for routine changes, whether shadow testing is warranted before a live test, and the A/B test design for a full rollout decision — including a pre-committed sample size, so nobody is tempted to peek and stop early.
- Hallucination/bias/safety measurement plan (Lesson 5): which metrics will be tracked ongoing, which are one-time pre-launch checks, and — critically — which categories of this specific feature are high-severity enough that a regression there should function as a near-automatic hard blocker in the ship/no-ship framework.
- Ship/no-ship criteria (Lesson 6): the actual thresholds and hard-blocker conditions that will be applied at the next launch review, stated concretely enough that two different people reading the plan would reach the same decision given the same dashboard.
The plan is worthless if the thresholds are vague
"We'll evaluate quality before launch" is not a plan -- it's a restatement of intent with the actual decision deferred to whoever happens to be in the room later. A real evaluation plan states numbers and conditions in advance: what hallucination rate on what category triggers a hold, what confidence interval width is required before an A/B result counts as trustworthy, what fairness gap on what protected characteristic is a hard blocker versus a monitor-and-report item. Writing these numbers down before you have a horse in the race (before a specific launch is on the calendar and a specific number is inconveniently close to the line) is what makes the plan a genuine commitment device instead of a document that gets quietly reinterpreted under launch pressure.
Building the rubric and running a sample scoring pass
A plan without a working rubric is theory. This project requires you to actually build the rubric (following Lesson 2's format: named criteria, weights, anchors, pass/fail rule) and run it against a sample of realistic outputs for your feature — at least 8 examples, scored the way a real reviewer would, including at least one example that should fail your own pass/fail rule. If every sample example passes easily, the rubric probably isn't anchored strictly enough to catch a real problem; a good rubric should be capable of failing a plausible bad output, not just validating good ones.
A learner's sample scoring pass shows all 8 example outputs scoring above 90% with no failures, and the rubric's pass/fail rule was never triggered. What should this prompt the learner to check?
A rubric that never fails anything on its own sample set hasn't actually been tested -- it's only been shown examples designed to pass. The diagnostic move is to check whether the sample deliberately includes at least one plausible bad output (a fabrication, a dropped edge case, a wrong attribution, the kinds of known failure modes from Lesson 1's golden dataset categories) and confirm the rubric's anchors and pass/fail rule actually catch it. If a deliberately flawed example still scores 90%+, the anchors are too lenient or the pass/fail rule's threshold is set too low to do real work in production.
Connecting forward: why this feeds the rest of the course
This isn't a standalone exercise. Phase 7's responsible AI review will take the hallucination/bias/safety measurement plan you write here and ask you to go deeper on consequence-scanning and mitigation — it assumes this evaluation plan already exists and is specific. The Phase 9 capstone assembles this evaluation plan alongside your Phase 2 opportunity assessment and Phase 4 PRD into one coherent portfolio case study, and later asks you to present it in a mock interview format. An evaluation plan built vaguely here creates real rework two phases later; one built with concrete, specific thresholds now is reusable with only incremental updates.
A learner is tempted to write a generic evaluation plan template with placeholder thresholds ("TBD based on data science input") to save time now, planning to fill in specifics later before the Phase 9 capstone. What is the strongest reason not to do this?
Writing 'TBD' defers the actual PM judgment call -- what hallucination rate is acceptable, what fairness gap is a hard blocker -- to some future moment, which is exactly the deferred-decision failure mode the callout above warns against. Doing that work now, while reasoning through this specific feature's risk profile with no launch pressure attached, produces a plan that's actually reusable in Phase 7's responsible AI review and the Phase 9 capstone. A vague plan doesn't save time; it just moves the same work later, under more time pressure, with the added cost of redoing analysis that could have been done once, carefully, here.