Glossary › Infrastructure

Inference

Running a trained model to produce outputs. Distinct from training. Optimizations: quantization, batching, KV cache, speculative decoding. Cost measured in $/M tokens.

Where this is taught

• Model Inversion & Membership Inference Attacks — Phase 3: Adversarial Machine Learning
• Inference Optimization — Phase 3: LLM Architecture
• KV Cache, Flash Attention & Inference Optimization — Phase 1: Transformer Architecture
• Natural Language Inference — Textual Entailment — Phase 3: Transformers for NLP
• Async and Hogwild! Inference — Phase 4: Training LLMs
• Model Serving & Inference Optimization — Phase 9: LLMOps & Production Deployment

Related terms

Quantization · KV Cache

More in Infrastructure

Vector Store · ANN (Approximate Nearest Neighbor) · Quantization · vLLM · Speculative Decoding

Learn this by building

DeVenture Academy teaches AI engineering through projects — every lesson is paired with a hands-on lab you run on your own machine with real tools.

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary