Phase 7: Voice Agent Frameworks · 40 min · Python

Framework Selection: Choosing the Right Tool for the Job

The framework you choose determines your control surface, speed-to-production, and cost structure. Choose systematically.

What you will learn

Develop a systematic framework selection decision tree
Compare Pipecat, LiveKit, OpenAI Realtime, Vapi, Retell, Bland on key criteria
Evaluate hybrid approaches using multiple frameworks
Map use case requirements to framework recommendations

The Problem

You've seen Pipecat, LiveKit, OpenAI Realtime API, Vapi, Retell AI, and Bland AI. Each has strengths and weaknesses. Choosing wrong means wasted development time, higher costs, or hitting limitations that force a rewrite.

The Concept

Decision Framework

Comparison Matrix

Framework	Setup	Cost/min	Custom LLM	Telephony	WebRTC	Open Source
Pipecat	Hard	$0.05	Yes	Plugins	No	Yes
LiveKit	Medium	$0.06	Yes	SIP Bridge	Yes	Yes
OpenAI Realtime	Easy	$0.06	No	None	No	No
Vapi	Easy	$0.10	Yes	Included	No	No
Retell AI	Easy	$0.10	Yes	Included	No	No
Bland AI	Easiest	$0.12	No	Included	No	No

You're a startup building a voice agent for browser-based customer support. You need WebRTC, want to use Claude as the LLM, and have a small budget. Which framework should you choose?

LiveKit is the right choice: (1) WebRTC-first — native browser support with built-in echo cancellation and NAT traversal, essential for browser-based support. (2) Custom LLM — LiveKit's plugin system supports any LLM, including Claude via livekit.plugins.anthropic or a custom plugin. (3) Budget — at $0.06/min (LiveKit Cloud) or free if self-hosted, it's affordable for startups. OpenAI Realtime API is wrong because it doesn't support Claude and has no WebRTC. Pipecat lacks WebRTC. Vapi/Retell/Bland are telephony-focused, not browser-optimized.

Selection Criteria

Criterion	Weight	Pipecat	LiveKit	Realtime API	Vapi	Retell	Bland
Setup speed	20%	2/10	6/10	9/10	8/10	8/10	10/10
Cost efficiency	20%	10/10	8/10	7/10	5/10	5/10	3/10
Customization	15%	10/10	8/10	3/10	7/10	8/10	4/10
WebRTC support	10%	3/10	10/10	5/10	2/10	2/10	2/10
Telephony	10%	7/10	5/10	1/10	9/10	9/10	9/10
Barge-in quality	10%	8/10	9/10	8/10	8/10	7/10	6/10
Community/docs	10%	6/10	8/10	9/10	7/10	6/10	5/10
Production ready	5%	8/10	9/10	7/10	8/10	8/10	7/10

Scoring Formula

def score_framework(framework, weights):
    """Score a framework based on weighted criteria."""
    total = 0
    for criterion, weight in weights.items():
        score = framework["scores"][criterion]
        total += score * weight
    return total / 100  # Normalize to 0-10

Migration Path

From	To	When to Migrate	Migration Effort
Bland AI	Vapi	Need custom LLM	Low (API change)
Vapi	Pipecat	Cost > $5K/month	Medium (rewrite pipeline)
Realtime API	Pipecat	Need non-OpenAI LLM	Medium (add ASR/TTS)
Pipecat	LiveKit	Need WebRTC	Medium (change transport)
Retell	Pipecat	Need full control	High (self-host everything)

Team Considerations

Team Profile	Recommended Framework	Why
Solo dev, no infra	Bland AI	Easiest, no servers
Small team, fast proto	Vapi or Retell	Managed, quick setup
Engineering team, production	Pipecat or LiveKit	Control, cost, open-source
Enterprise, compliance	Retell (custom LLM)	On-prem LLM, white-label
Browser-first product	LiveKit	WebRTC native
OpenAI shop, simple	Realtime API	One provider, one connection

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.