Glossary › Infrastructure

Quantization

Reducing model precision from FP16 to INT8, INT4, or lower. Reduces memory and speeds up inference with minimal quality loss. Methods: GPTQ, AWQ, GGUF, bitsandbytes. Essential for local deployment.

Where this is taught

• Model Optimization: Quantization & Distillation — Phase 9: LLMOps & Production Deployment
• Quantization: Making Models Fit — Phase 3: LLM Architecture
• GPU Resource Management — Phase 8: Serverless GPU Deployment & Infrastructure
• Local LLMs & Open-Weight Models — Phase 5: Modern LLM Engineering
• The Local AI Stack — Design and Hardware Check — Build a Local AI Dev Environment
• Performance Optimization & Scaling — Phase 8: Production Hardening & Operations

Related terms

Inference · QLoRA

More in Infrastructure

Vector Store · ANN (Approximate Nearest Neighbor) · Inference · vLLM · Speculative Decoding

Learn this by building

DeVenture Academy teaches AI engineering through projects — every lesson is paired with a hands-on lab you run on your own machine with real tools.

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary