Phase 4: Training LLMs · ~75 minutes · Python

Speculative Decoding and EAGLE-3

Phase 7 · Lesson 16 proved the math: the Leviathan rejection rule preserves the verifier's distribution exactly. This lesson is the training-stack view of 2026 production speculative decoding. EAGLE-3 turned the draft model from a cheap approximation into a purpose-built tiny network trained on the verifier's own hidden states, then added a training-time test loop that aligns its train and inference distributions. Result: 3× to 6.5× end-to-end speedup, accepted per-token rates above 0.9 on chat, no distributional tradeoff. Every production inference stack in 2026 ships it by default.

Hiring signal: Understanding of speculative decoding and eagle-3 internals

What you will learn

State the Leviathan theorem in one sentence and prove that the speculative loop produces samples identically distributed to the verifier.
Walk the two-year progression from vanilla spec-decoding (Leviathan 2023) through EAGLE, EAGLE-2, and EAGLE-3 and name the exact limitation each step removed.
Compute expected speedup from acceptance rate `α` and draft-to-verifier cost ratio `c`, and choose the optimal draft length `N` for each regime.
Implement the full speculative loop from scratch: draft, verify, reject-sample from the residual, roll the KV cache back on rejection, emit the bonus token on full acceptance.

Introduction

Type: Build Languages: Python (stdlib) Prerequisites: Phase 7 · 16 (speculative decoding math), Phase 10 · 12 (inference optimization) Time: ~75 minutes

Learning Objectives

State the Leviathan theorem in one sentence and prove that the speculative loop produces samples identically distributed to the verifier.
Walk the two-year progression from vanilla spec-decoding (Leviathan 2023) through EAGLE, EAGLE-2, and EAGLE-3 and name the exact limitation each step removed.
Compute expected speedup from acceptance rate α and draft-to-verifier cost ratio c, and choose the optimal draft length N for each regime.
Implement the full speculative loop from scratch: draft, verify, reject-sample from the residual, roll the KV cache back on rejection, emit the bonus token on full acceptance.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.