Phase 8: Production Agent Infrastructure · 90 min · Python · Anthropic SDK · HuggingFace TRL

Fine-Tuning Agents for Tool Use

A fine-tuned 8B model that knows your exact tools will outperform a general 70B model that's seeing them for the first time.

Hiring signal: Fine-tuning for specific tasks is a clear differentiator. Companies with production agent systems often reach a ceiling with general models and invest in fine-tuning. Engineers who can own the full pipeline — data generation, training, evaluation, serving — are rare and valuable, particularly at AI-first companies building proprietary models.

What you will learn

Generate a synthetic tool call training dataset using self-instruct patterns and LLM teacher models
Evaluate tool call accuracy on three metrics: tool selection, schema adherence, and argument type correctness
Understand the SFT + DPO training pipeline structure using HuggingFace TRL

The Problem

Your general-purpose model calls the wrong tool 15% of the time. It misformats tool arguments 8% of the time. It confuses process_refund and escalate_to_human for ambiguous queries. These error rates compound: in a 5-step agent workflow, a 15% per-step error rate means you complete correctly only 44% of the time.

The fix is not a better prompt. You've already tried that. The fix is fine-tuning: teaching a smaller model to deeply know your specific tools, argument schemas, and the mapping from user intents to correct tool calls. After fine-tuning on 500 examples, tool selection accuracy goes from 85% to 97%+. Schema adherence goes from 60% to 98%+.

And the fine-tuned 8B model costs 1/8 as much to run as the general 70B model you were relying on. This is the production case for fine-tuning: higher accuracy, lower cost, for stable and well-defined tool sets.

When to fine-tune vs. prompt engineer
Fine-tuning makes sense when: your tool set is stable (not changing weekly), you need consistent accuracy across high volumes (>10K tool calls/day), and a general model isn't reaching your accuracy target with few-shot prompting. Don't fine-tune if you're still iterating on your tools — every schema change requires regenerating data and retraining. Prompting is cheaper to iterate; fine-tuning is cheaper to operate at scale.

Why Fine-Tune for Tool Use

General models learn tool use from the tool schemas and few-shot examples in the prompt. This works — to a point. The model has never seen your specific tools before the prompt, and it's generalizing from a small number of examples.

Fine-tuned models have it baked in. Benefits:

Higher accuracy: fewer wrong tool selections, fewer malformed arguments. The model has seen thousands of examples of correct tool calls and doesn't need to generalize from a few.

Smaller prompt: you need fewer examples in the prompt (smaller system prompt). The fine-tuned model already knows the patterns.

Consistency: the fine-tuned model applies tools more consistently across diverse user inputs, including edge cases, multilingual queries, and unusual phrasings.

Cost efficiency: a fine-tuned Llama 3.2 3B model running on a $0.50/hour A10G GPU may match a general Claude 3.5 Sonnet on your specific tool calling task at 1/100th the per-call cost at volume.

A fine-tuned 8B model with 5,000 tool call examples vs. a general 70B model with 10 few-shot examples in the prompt. Which is likely better at tool calling?

Explanation: This is the core fine-tuning vs. prompting tradeoff. Fine-tuning is expensive upfront (training cost, data generation) but produces a model that deeply knows your specific tools. Few-shot prompting is flexible but relies on the model generalizing from a small context window. For production systems with stable, well-defined tool sets processing high volumes, a fine-tuned small model consistently outperforms general large models on tool accuracy — and costs a fraction as much per call.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Synthetic Dataset Generation, Training and Evaluation Pipeline, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.