Phase 1: AI & ML Literacy for PMs · 45 min · OpenAI Fine-tuning Docs · Python
RAG, Fine-Tuning & Agents — When to Use Which
RAG answers 'what does the model know right now.' Fine-tuning answers 'how should the model behave.' Agents answer 'what should the model do next.' Confusing the three burns a quarter of engineering time on the wrong build.
Hiring signal: AI PM interview loops explicitly test whether a candidate can reason about RAG vs fine-tuning vs agentic approaches with real cost/complexity tradeoffs, not just buzzword familiarity — it's one of the four core signals evaluators screen for. A PM who defaults to 'let's fine-tune it' or 'let's make it an agent' without first asking what specific gap each approach closes is the PM whose roadmap commitment gets reopened in the next planning cycle.
What you will learn
- Explain what RAG, fine-tuning, and agentic approaches each actually change about model behavior, in plain language
- Map a product need (current knowledge, consistent behavior/format, multi-step task execution) to the correct approach using a decision framework
- Compare the cost and engineering complexity of RAG, fine-tuning, and agents, including where they can be combined
- Identify when a product need is being over-engineered — e.g., reaching for an agent when a single prompt would do
- Recommend and defend an approach for a given scenario using the tradeoffs covered in this lesson, not vendor marketing
The Problem
A product team wants to build "ask questions about our internal engineering wiki." In the kickoff meeting, three people propose three different builds. The ML engineer wants to fine-tune an open-weight model on the wiki's full text, arguing "then the model will just know it." The founding engineer wants to build a multi-step agent that can search the wiki, cross-reference Jira tickets, and post a summary to Slack, because "agents are what everyone's building now." The PM, unsure which is right, agrees to a timeline that tries to accommodate a version of all three.
Eight weeks later: the fine-tune is stale because the wiki gets edited weekly and nobody wants to re-run a training job every week. The agent's multi-step Jira-cross-referencing capability is barely used — most questions are single-lookup, not multi-step tasks. The team ships a much smaller RAG pipeline (retrieve relevant wiki pages, pass them to the model, answer the question) in the final two weeks, and it's the part of the build that actually works.
This is not a story about bad engineers. It's a story about skipping the one question that should have been asked in the kickoff meeting: what, specifically, is the gap between what the model can already do and what this feature needs? RAG, fine-tuning, and agents close three genuinely different gaps. Reaching for the wrong one doesn't just waste budget — it ships a feature that's more complex than the problem required, and complexity is where AI features go to fail quietly.
What Each Approach Actually Changes
RAG (retrieval-augmented generation) doesn't change the model at all. It changes what the model sees at the moment it answers: instead of relying on what's baked into the model's weights from training, RAG retrieves relevant chunks of your own data (documents, tickets, product data) and inserts them into the prompt as context. The model then answers grounded in that retrieved content. RAG is the right tool when the gap is knowledge — the model doesn't know something specific to you, and that something changes often enough that baking it into weights would go stale.
Fine-tuning changes the model's weights using examples you provide. It doesn't reliably teach new facts (as covered in the previous lesson on build vs buy) — it changes how the model responds: the format, tone, structure, or behavior pattern it defaults to for a given kind of input. Fine-tuning is the right tool when the gap is consistency of behavior — you need the model to reliably produce a specific output shape or voice across thousands of similar cases, and prompting alone hasn't gotten it reliable enough.
Agents don't change what the model knows or how it's tuned — they change what the model is allowed to do. An agentic system gives the model access to tools (search, code execution, API calls) and lets it decide, across multiple steps, which tools to call and in what order to accomplish a task. Agents are the right tool when the gap is multi-step task execution — the job genuinely requires deciding a sequence of actions based on intermediate results, not just answering a single question.
These three are not mutually exclusive. A production system often combines them: an agent that uses RAG as one of its tools, running on a model that's been fine-tuned to format its final answers a specific way. The mistake in the opening scenario wasn't using any one of these — it was reaching for all three before establishing which gap the feature actually had.
Start from the gap, not the technique
Before proposing RAG, fine-tuning, or an agent, answer one question: is the problem that the model doesn't know something (RAG), that its behavior/format isn't consistent enough (fine-tuning), or that the task requires multiple decided steps rather than one answer (agent)? If you can't name which gap you're closing, you're choosing a technique because it's trendy, not because it fits — and that's exactly how a team ends up building three things when the problem needed one.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Cost and Complexity, Roughly Ordered, When Each Approach Is Over-Engineering, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy