Phase 8: Evaluation, Testing & Production · 50 min · Python · AssemblyAI SDK · Deepgram SDK
The Concept
Post-Call Analytics Pipeline
| Analysis | Input | Output | Tool |
|---|
| Transcription | Audio recording | Full text transcript | Whisper, Deepgram |
| Summarization | Transcript | 2-3 sentence summary | GPT-4o-mini |
| Sentiment | Transcript | Positive/Neutral/Negative + score | LLM or NLP model |
| Action items | Transcript | List of follow-up tasks | LLM extraction |
| Quality score | Transcript + metrics | 0-100 score | Custom scoring |
| Topic detection | Transcript | Categories/tags | LLM classification |
| Compliance check | Transcript | Pass/fail per rule | Rule engine + LLM |
Why is post-call sentiment analysis more valuable than real-time sentiment?
Post-call analyzes the full conversation — you see the complete emotional arc (frustrated → helped → satisfied), not just a snapshot. You can correlate sentiment with outcomes (negative sentiment + escalation = training opportunity)
Transcription
class CallTranscriber:
def __init__(self):
self.transcript = []
async def transcribe(self, audio_file):
"""Transcribe call audio to text."""
# Using Whisper or Deepgram
segments = await self._run_asr(audio_file)
for seg in segments:
self.transcript.append({
"speaker": seg["speaker"], # "agent" or "caller"
"text": seg["text"],
"start_time": seg["start"],
"end_time": seg["end"],
"confidence": seg["confidence"],
})
return self.transcript
Summarization
class CallSummarizer:
def __init__(self):
self.llm = OpenAIClient(model="gpt-4o-mini")
async def summarize(self, transcript):
"""Generate a call summary."""
conversation_text = self._format_transcript(transcript)
prompt = f"""Summarize this voice agent call in 2-3 sentences:
{conversation_text}
Include: caller's request, agent's response, outcome (resolved/escalated/abandoned).
"""
summary = await self.llm.complete(prompt)
return summary
Sentiment Analysis
class SentimentAnalyzer:
def __init__(self):
self.llm = OpenAIClient(model="gpt-4o-mini")
async def analyze(self, transcript):
"""Analyze sentiment across the conversation."""
conversation_text = self._format_transcript(transcript)
prompt = f"""Analyze the caller's sentiment in this conversation.
Return JSON: {{"overall": "positive/neutral/negative", "score": 0-10,
"arc": "improved/stable/declined", "key_moments": [...]}}
{conversation_text}
"""
result = await self.llm.complete(prompt)
return json.loads(result)
Action Item Extraction
class ActionItemExtractor:
def __init__(self):
self.llm = OpenAIClient(model="gpt-4o-mini")
async def extract(self, transcript):
"""Extract action items from the call."""
prompt = f"""Extract action items from this conversation.
Return JSON array: [{{"action": "...", "assignee": "agent/human/none",
"priority": "high/medium/low", "deadline": "..."}}]
{self._format_transcript(transcript)}
"""
result = await self.llm.complete(prompt)
return json.loads(result)
Quality Scoring
class QualityScorer:
def __init__(self):
self.llm = OpenAIClient(model="gpt-4o-mini")
async def score(self, transcript, metrics):
"""Score call quality on multiple dimensions."""
prompt = f"""Score this voice agent call on 5 dimensions (0-10 each):
1. Task completion: Did the agent resolve the caller's issue?
2. Communication clarity: Were responses clear and concise?
3. Tone appropriateness: Was the agent professional and empathetic?
4. Efficiency: Did the agent resolve the issue without unnecessary steps?
5. Error handling: How well did the agent handle any issues?
Return JSON: {{"scores": {{...}}, "overall": 0-10, "notes": "..."}}
{self._format_transcript(transcript)}
Metrics: {json.dumps(metrics)}
"""
result = await self.llm.complete(prompt)
return json.loads(result)
Analytics Dashboard Data
class PostCallAnalytics:
"""Complete post-call analytics pipeline."""
async def process_call(self, call_recording, call_metrics):
"""Process a completed call."""
# 1. Transcribe
transcript = await self.transcriber.transcribe(call_recording)
# 2. Summarize
summary = await self.summarizer.summarize(transcript)
# 3. Sentiment
sentiment = await self.sentiment.analyze(transcript)
# 4. Action items
actions = await self.action_extractor.extract(transcript)
# 5. Quality score
quality = await self.quality_scorer.score(transcript, call_metrics)
return {
"call_id": call_recording.call_id,
"transcript": transcript,
"summary": summary,
"sentiment": sentiment,
"action_items": actions,
"quality_score": quality,
"metrics": call_metrics,
}
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.