The Concept
Key Quality Metrics
| Metric | What It Measures | Target | Why It Matters |
|---|
| Talk Ratio | Agent vs caller speaking time | 40-60% agent | 70%+ = agent dominates |
| WPM (Words Per Minute) | Agent speaking speed | 140-160 WPM | >180 = too fast, <120 = too slow |
| Task Completion Rate | % of calls achieving goal | >85% | Low = agent fails |
| Interruption Rate | Agent interrupts caller per call | <0.5 | High = poor turn-taking |
| Silence Duration | Unexplained silence per call | <2s | Long silence = broken |
| Call Duration | Average call length | Varies | Too long = inefficient |
| Containment Rate | % resolved without human | >80% | Low = escalation too frequent |
| CSAT | Customer satisfaction score | >4.0/5 | Direct user feedback |
If a voice agent has a talk ratio of 75%, what does that indicate?
The agent is dominating the conversation — talking 75% of the time vs 25% for the caller. Ideal is 40-60% agent. 75% means the agent talks too much, likely giving long responses and not listening enough
Talk Ratio
class TalkRatioCalculator:
def __init__(self):
self.agent_speaking_time = 0
self.caller_speaking_time = 0
self.total_time = 0
def add_agent_speech(self, duration_s):
"""Add agent speaking duration."""
self.agent_speaking_time += duration_s
self.total_time += duration_s
def add_caller_speech(self, duration_s):
"""Add caller speaking duration."""
self.caller_speaking_time += duration_s
self.total_time += duration_s
def get_ratio(self):
"""Calculate talk ratio."""
if self.total_time == 0:
return 0
return self.agent_speaking_time / self.total_time * 100
WPM (Words Per Minute)
class WPMCalculator:
def __init__(self):
self.utterances = []
def add_utterance(self, text, duration_s):
"""Add an agent utterance with its duration."""
words = len(text.split())
wpm = (words / duration_s) * 60
self.utterances.append({
"text": text,
"words": words,
"duration_s": duration_s,
"wpm": wpm,
})
def get_average_wpm(self):
"""Get average WPM across all utterances."""
if not self.utterances:
return 0
return sum(u["wpm"] for u in self.utterances) / len(self.utterances)
Task Completion Rate
class TaskCompletionTracker:
def __init__(self, required_slots):
self.required_slots = required_slots # e.g., ["destination", "date", "passengers"]
self.filled_slots = {}
self.completed = False
self.abandoned = False
def fill_slot(self, slot_name, value):
"""Fill a required slot."""
self.filled_slots[slot_name] = value
if all(s in self.filled_slots for s in self.required_slots):
self.completed = True
def abandon(self):
"""Caller hung up before completing."""
self.abandoned = True
def is_complete(self):
"""Check if task is complete."""
return self.completed
Containment Rate
class ContainmentTracker:
def __init__(self):
self.total_calls = 0
self.contained_calls = 0 # Resolved without human
self.escalated_calls = 0
def record_call(self, escalated=False):
"""Record a call outcome."""
self.total_calls += 1
if escalated:
self.escalated_calls += 1
else:
self.contained_calls += 1
def get_containment_rate(self):
"""Calculate containment rate."""
if self.total_calls == 0:
return 0
return self.contained_calls / self.total_calls * 100
Quality Evaluation Pipeline
Quality Score Calculation
def calculate_quality_score(metrics):
"""Calculate overall quality score (0-100)."""
score = 0
# Talk ratio (25 points)
talk_ratio = metrics["talk_ratio"]
if 40 <= talk_ratio <= 60:
score += 25
elif 35 <= talk_ratio <= 65:
score += 15
else:
score += 5
# WPM (20 points)
wpm = metrics["wpm"]
if 140 <= wpm <= 160:
score += 20
elif 120 <= wpm <= 180:
score += 10
else:
score += 0
# Task completion (30 points)
score += metrics["task_completion_rate"] * 0.3
# Containment (15 points)
score += metrics["containment_rate"] * 0.15
# Interruption rate (10 points)
if metrics["interruption_rate"] < 0.5:
score += 10
elif metrics["interruption_rate"] < 1.0:
score += 5
return round(score, 1)
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.