Phase 4: LLM Orchestration for Voice · 55 min · Python · OpenAI SDK · Anthropic SDK

Streaming LLM Output and System Prompt Design for Voice

Write for the ear, not the eye. Spoken prompts are shorter, simpler, and more natural.

Hiring signal: Voice-specific prompt engineering distinguishes voice AI engineers from text AI engineers.

What you will learn

Implement streaming LLM output where token-by-token generation feeds directly into TTS
Design system prompts optimized for spoken conversation: concise, turn-aware, spoken-language
Write for the ear vs the eye: shorter sentences, contractions, natural pauses
Handle numbers, dates, and special terms in spoken output

The Problem

Text chatbot prompts produce long, formatted responses with markdown and bullet points. Voice agents need the opposite: short, natural, spoken-language responses. The system prompt is the single most impactful lever for voice agent quality.

The Concept

Voice vs Text System Prompts

Aspect	Text Chatbot	Voice Agent
Response length	Unlimited	1-3 sentences (under 30 words)
Formatting	Markdown, lists, bold	Plain text only
Tone	Can be formal	Conversational, natural
Numbers	"1,247.89"	"twelve forty seven"
Pauses	N/A	Natural sentence breaks
Emojis	Yes	Never
Questions	Can ask multiple	One at a time

The Voice Agent System Prompt

VOICE_SYSTEM_PROMPT = """You are a voice agent for [Company Name].

CORE RULES:
1. Keep responses to 1-3 sentences. Never exceed 30 words unless reading back data.
2. Speak naturally — this is a phone call, not a text message.
3. Never use markdown, bullet points, emojis, or formatting.
4. Don't repeat what the caller said. Just act on it.
5. Ask one question at a time. Wait for the answer.
6. Use simple words. This is spoken, not written.
7. For numbers, say them naturally: "two fifty" not "two hundred and fifty dollars"
8. If you need to check something, call the tool and say "Let me check that for you."
9. If the caller is frustrated, acknowledge it before solving.
10. Never say "as an AI", "I'm a language model", or "I don't have feelings."

PERSONA:
- You are [friendly/professional/casual] and [helpful/efficient/empathetic].
- You work for [Company Name], a [company description].
- You can help with [list of capabilities].

TOOLS:
- check_balance: Look up a customer's account balance
- book_flight: Search and book flights
- transfer_call: Escalate to a human agent

When you don't know something, say "Let me connect you with someone who can help."
"""

Why must voice agent responses be limited to 1-3 sentences?

TTS can't handle long text

Streaming LLM → TTS Pipeline

The key optimization: don't wait for the full LLM response. Pipe tokens to TTS as soon as a complete sentence is formed.

Sentence Boundary Detection

def split_at_sentence_boundaries(token_stream):
    """Buffer tokens and yield complete sentences for TTS."""
    buffer = ""
    for token in token_stream:
        buffer += token
        # Check for sentence-ending punctuation
        if any(punct in buffer for punct in ".!?"):
            yield buffer.strip()
            buffer = ""
    if buffer:  # Flush remaining
        yield buffer.strip()

Response Length Control

Technique	How	Effect
System prompt rules	"Keep responses to 1-3 sentences"	LLM self-limits
max_tokens=150	API parameter	Hard limit on output length
Temperature 0.7	API parameter	Natural variation, not robotic
Few-shot examples	Show short Q&A pairs	LLM mimics the pattern
Stop sequences	["\n\n"]	Prevent paragraph breaks

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.