Glossary › Infrastructure

Speculative Decoding

Inference optimization where a small draft model proposes tokens that the large model verifies in parallel. Reduces latency by 2-3× for greedy decoding. No quality loss.

Where this is taught

• Speculative Decoding and EAGLE — Phase 4: Training LLMs
• Speculative Decoding — Draft, Verify, Repeat — Phase 2: Pre-training & Fine-tuning
• Speculative Decoding and EAGLE-3 — Phase 4: Training LLMs
• Inference Optimization — Phase 3: LLM Architecture
• Performance Optimization & Scaling — Phase 8: Production Hardening & Operations
• Multi-Token Prediction (MTP) — Phase 4: Training LLMs

Related terms

Inference

More in Infrastructure

Vector Store · ANN (Approximate Nearest Neighbor) · Inference · Quantization · vLLM

Learn this by building

DeVenture Academy teaches AI engineering through projects — every lesson is paired with a hands-on lab you run on your own machine with real tools.

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary