Glossary › Infrastructure

Speculative Decoding

Inference optimization where a small draft model proposes tokens that the large model verifies in parallel. Reduces latency by 2-3× for greedy decoding. No quality loss.

Where this is taught

• EAGLE-3 Speculative Decoding in Production — Phase 1: Serving Engine Internals
• Speculative Decoding and EAGLE — Phase 4: Training LLMs
• Speculative Decoding — Draft, Verify, Repeat — Phase 2: Pre-training & Fine-tuning
• Speculative Decoding and EAGLE-3 — Phase 4: Training LLMs
• Speculative-Decoding Inference Server — Phase 2: Models, Serving & Observability
• Inference Optimization — Phase 3: LLM Architecture

Related terms

Inference

More in Infrastructure

Vector Store · ANN (Approximate Nearest Neighbor) · Inference · Quantization · vLLM

Learn this by building

DeVenture Academy teaches AI engineering through projects, with hands-on labs you run on your own machine using real tools.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary