Glossary › Deployment

Streaming

Returning tokens as they are generated rather than waiting for the full response. Improves perceived latency (TTFT). Implemented via SSE (Server-Sent Events) or WebSocket.

Where this is taught

• Streaming TTS: Why First-Byte Latency Matters — Phase 3: Text-to-Speech (TTS)
• Streaming vs Batch ASR: Partial vs Final Transcripts — Phase 2: Speech-to-Text (ASR)
• WebSocket Audio Streaming: The OpenAI Realtime API Pattern — Phase 6: Real-Time Transport & Telephony
• Streaming LLM Output and System Prompt Design for Voice — Phase 4: LLM Orchestration for Voice
• WebSocket Audio Streaming Server — Phase 1: Audio Processing & Streaming
• Pipecat: Frame-Based Streaming Pipelines — Phase 7: Voice Agent Frameworks

Related terms

Latency · Inference

More in Deployment

Latency · Throughput

Learn this by building

DeVenture Academy teaches AI engineering through projects, with hands-on labs you run on your own machine using real tools.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary