Phase 3: LLM Architecture · ~90 minutes · Python
Instruction Tuning (SFT)
A base model predicts the next token. That's it. It doesn't follow instructions, answer questions, or refuse harmful requests. SFT is the bridge between a token predictor and a useful assistant. Every model you've ever talked to -- Claude, GPT, Llama Chat -- went through this step.
Hiring signal: Understanding of instruction tuning (sft) internals
What you will learn
- Implement supervised fine-tuning (SFT) that converts a base language model into an instruction-following assistant
- Format training data using chat templates with system, user, and assistant roles, and mask loss on non-assistant tokens
- Explain why SFT is necessary: base models continue text rather than answer questions
- Evaluate SFT quality by comparing base model vs fine-tuned model responses on a held-out instruction set
Introduction
Type: Build Languages: Python (with numpy) Prerequisites: Phase 10, Lesson 04 (Pre-Training a Mini GPT) Time: ~90 minutes
Learning Objectives
- Implement supervised fine-tuning (SFT) that converts a base language model into an instruction-following assistant
- Format training data using chat templates with system, user, and assistant roles, and mask loss on non-assistant tokens
- Explain why SFT is necessary: base models continue text rather than answer questions
- Evaluate SFT quality by comparing base model vs fine-tuned model responses on a held-out instruction set
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy