Phase 2: Speech-to-Text (ASR) · 40 min · Python · Deepgram SDK · AssemblyAI SDK

Streaming vs Batch ASR: Partial vs Final Transcripts

Streaming partials are what make voice agents feel real. Batch is for after the call.

Real-world application: Understanding streaming vs batch ASR and partial transcripts is essential for voice agent engineering.

What you will learn

Explain the difference between streaming and batch ASR
Understand partial vs final transcripts and how partials update
Identify when to use streaming (voice agents) vs batch (post-call analysis)
Reason about the streaming latency vs accuracy trade-off

The Problem

In a voice agent, ASR sends partial transcripts as the caller speaks. "I want to book a..." arrives as:

Frame 10: "I"
Frame 20: "I want"
Frame 30: "I want to"
Frame 40: "I want to book"
Frame 50: "I want to book a flight" (final)

If you send the LLM on "I want to book a", you'll get a response about booking "a" (something). If you wait too long, you add latency. Knowing when to act is the endpointing problem — and it starts with understanding partial vs final transcripts.

The Concept

Streaming ASR Event Types

Event	Description	When to Act
`Partial`	Interim transcript, may change	Display only, don't act
`Final`	Complete utterance, won't change	Send to LLM
`SpeechFinal`	Speaker has finished (Deepgram)	Trigger LLM + TTS
`UtteranceEnd`	Silence detected, utterance complete	Alternative trigger

How Partials Work

Why shouldn't you send partial transcripts to the LLM?

Partials update as more audio arrives. "I want to book a" will likely become "I want to book a flight to Tokyo" — acting on the partial means the LLM responds to an incomplete request. The response would be generic ("What would you like to book?") instead of specific ("I can help you book a flight to Tokyo"). Wait for the final transcript (triggered by silence detection / SpeechFinal) before sending to the LLM. The exception: use partials for real-time UI display ("typing indicator" equivalent).

Batch ASR

Batch ASR processes the complete audio file at once:

Input: Full audio file (WAV, MP3)
Output: Complete transcript with timestamps
Latency: 2-30 seconds (proportional to audio length)
Use cases: Post-call transcription, voicemail, meeting recording

Streaming ASR

Streaming ASR processes audio in real-time:

Input: Audio chunks (20ms frames via WebSocket)
Output: Partial transcripts → final transcript
Latency: 100-300ms TTFT
Use cases: Voice agents, live captioning, real-time commands

When to Use Each

Use Case	Mode	Why
Voice agent (real-time conversation)	Streaming	Need partials for UI + finals for LLM
Post-call transcription	Batch	Better accuracy, no latency constraint
Voicemail to text	Batch	Audio is complete, accuracy matters
Meeting recording	Batch	Full audio available, need diarization
Live captioning	Streaming	Must display text as people speak
Voice command (short)	Streaming	Need immediate response

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.