Phase 3: LLM Architecture · ~90 minutes · Python

Instruction Tuning (SFT)

A base model predicts the next token. That's it. It doesn't follow instructions, answer questions, or refuse harmful requests. SFT is the bridge between a token predictor and a useful assistant. Every model you've ever talked to -- Claude, GPT, Llama Chat -- went through this step.

What you will learn

Implement supervised fine-tuning (SFT) that converts a base language model into an instruction-following assistant
Format training data using chat templates with system, user, and assistant roles, and mask loss on non-assistant tokens
Explain why SFT is necessary: base models continue text rather than answer questions
Evaluate SFT quality by comparing base model vs fine-tuned model responses on a held-out instruction set

Introduction

Type: Build
Languages: Python (with numpy)
Prerequisites: Pre-Training a Mini GPT (124M Parameters)
Time: ~90 minutes

Objective

Learning objectives

Implement supervised fine-tuning (SFT) that converts a base language model into an instruction-following assistant
Format training data using chat templates with system, user, and assistant roles, and mask loss on non-assistant tokens
Explain why SFT is necessary: base models continue text rather than answer questions
Evaluate SFT quality by comparing base model vs fine-tuned model responses on a held-out instruction set

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.