Glossary › Deployment

Latency

Time from request to first token (TTFT) or full response. TTFT depends on prompt processing. Token generation rate (tokens/s) determines total latency. Optimized by batching, quantization, speculative decoding.

Where this is taught

• Cost & Latency Budgeting in the PRD — Phase 4: Writing AI Product Requirements
• Latency Metrics: Glass-to-Glass, Per-Component, P50/P95/P99 — Phase 8: Evaluation, Testing & Production
• The Latency Budget and Cost Economics — Phase 0: Voice AI Fundamentals & Architecture
• Streaming TTS: Why First-Byte Latency Matters — Phase 3: Text-to-Speech (TTS)
• Model Selection for Voice Agents — Phase 4: LLM Orchestration for Voice
• Model Selection Tradeoffs — Phase 2: Discovery & AI Product Strategy

Related terms

Inference · Throughput

More in Deployment

Throughput · Streaming

Learn this by building

DeVenture Academy teaches AI engineering through projects, with hands-on labs you run on your own machine using real tools.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary