Phase 4: Multi-Agent Systems · 60 min · Python · Anthropic SDK
Reflection & Self-Critique Agents
Single-pass outputs are a first draft, not a final answer.
Hiring signal: Reflection loops are used in production at companies running AI writing assistants, code review tools, and automated analysis pipelines. The ability to implement reliable self-critique with proper termination conditions and cost awareness is a sign of engineering maturity — not just pattern-following.
What you will learn
- Implement a reflection loop: generate → critique → revise → repeat until passing
- Design a critique prompt that produces actionable, specific feedback
- Apply self-consistency: run the task multiple times and vote on the best output
The Problem
Single-pass generation is reliable for simple tasks, but as complexity grows, the first output is rarely the best. The model doesn't know what it doesn't know. A reflection loop gives the model a second perspective on its own work: generate, step back and critique, revise based on specific feedback, repeat until the output meets a quality threshold.
The insight from the Reflexion paper (Shinn et al., 2023): verbal feedback from a critic agent is a surprisingly effective form of learning signal. The model uses the critique to reason about its mistakes and generate better outputs.
The Reflection Pattern
The loop has three roles, each with a distinct purpose:
Generator: produces the initial output. Can be the same model with a "generate" system prompt, or a separate model.
Critic: evaluates the output against specific quality dimensions. Must produce structured, actionable feedback — not vague praise or complaints.
Reviser: takes the original output + the critic's feedback and produces an improved version. Uses the critique as an explicit set of instructions.
The critic's passes field is the termination condition. When the critic says passes: true, the loop exits. The hard revision limit (max_revisions=3) is the safety net.
Your reflection loop runs the critic 5 times but the output never "passes." What safeguard must you have?
Reflection loops without a revision limit can run indefinitely, consuming tokens and time. In production, set max_revisions=3. After 3 cycles, return the output at its current quality state — potentially with a flag that human review is recommended. The goal is 'better than single-pass,' not 'perfect.'
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Designing the Critique Prompt, Self-Consistency and Voting, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy