Phase 4: Writing AI Product Requirements · 45 min · Markdown · Python
Designing for Failure
A confident wrong answer is a worse product experience than a slow right one — design for the failure, not just the happy path.
Hiring signal: AI PM interview loops explicitly probe 'how you ship responsibly under uncertainty' as a core signal — and failure-state design is where that shows up as a concrete deliverable rather than an abstract value. A candidate who can walk through a specific escalation-tier design, with named confidence thresholds and named human-review triggers, answers that interview question with an artifact instead of a platitude.
What you will learn
- Design hallucination UX patterns: citations, confidence framing, and uncertainty disclosure
- Specify abstention behavior — when and how a feature should say 'I don't know' instead of guessing
- Design human-in-the-loop escalation tiers keyed to specific confidence or risk thresholds
- Produce a failure-state design spec covering error states, abstention, and escalation for a real AI feature
The Problem
An AI legal-research assistant confidently cites a case: "As established in Miller v. Thompson (2019), the statute of limitations for this claim is four years." A junior associate uses it in a client memo. The case doesn't exist — the model generated a plausible-sounding citation with no basis in its training data or any retrieval step, because nothing in the product ever told it that inventing a citation was worse than saying "I couldn't find a directly applicable precedent." The PRD for this feature had a full section on output formatting and tone. It had zero sentences about what the system should do when it doesn't actually know the answer. That omission wasn't a technical gap engineering forgot to fill — it was a product decision nobody made, so the model made the only decision it's statistically inclined to make by default: produce something confident and complete-shaped, because that's what its training rewards, absent an explicit instruction otherwise.
This is the core insight this lesson builds on: failure is not an edge case for a generative feature, it's a first-class product surface that needs the same design attention as the happy path. A deterministic feature fails loudly and obviously — a null pointer exception, a 500 error, a form that won't submit. A generative feature's most dangerous failure mode is the opposite: it fails quietly and confidently, producing something that looks exactly like a correct answer. Designing for failure means designing the specific mechanisms that make failure visible, contained, and escalatable instead of indistinguishable from success.
Hallucination UX Patterns
Three patterns, usable individually or combined, make hallucination risk visible to the user instead of hidden inside confident prose:
- Citations/grounding — every factual claim links back to a specific source (a retrieved document, a specific line in a transcript, a database record). This doesn't prevent hallucination, but it makes it checkable: a user or reviewer can click through and verify, and a claim with no citation is itself a signal something's off.
- Confidence framing — language that reflects actual certainty instead of uniformly confident prose. "Based on the documents provided, it appears..." reads differently than a bare assertion, and that difference should track something real (retrieval quality, model-reported uncertainty where available) rather than being decorative hedging applied uniformly regardless of actual confidence.
- Uncertainty disclosure at the UI level — a visible indicator (not just careful wording) when the system is operating outside its comfortable zone: low retrieval match confidence, a question type the system historically gets wrong, or output that couldn't be verified against a grounding source.
Hedging language only works if it's calibrated — uniform hedging is worse than no hedging
If every single response says "I believe" or "it appears," those phrases stop carrying information and become filler users learn to ignore, exactly like a smoke detector that beeps constantly gets its battery pulled. The hedge has to be conditional — present when confidence is actually lower, absent when it's actually higher — which means the product needs some real signal driving it (retrieval match score, self-reported model confidence, whether a citation was found) rather than a stylistic instruction to "sound a bit uncertain sometimes." Design the confidence signal first; the UI language is just how that signal gets communicated.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Abstention: Designing "I Don't Know", Human-in-the-Loop Escalation Tiers, Error-State Design Beyond the Model, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy