Phase 5: Turn-Taking & Conversation Management · 45 min · Python · Vapi SDK · Retell AI SDK

Escalation to Human and Multi-Agent Handoffs

Knowing when to escalate is as important as knowing how to handle the call. Build both.

Hiring signal: Escalation and handoff implementation shows production awareness and multi-agent architecture skills.

What you will learn

Detect when to escalate: complex requests, emotional callers, compliance requirements
Implement warm and cold transfers with context preservation
Build multi-agent handoffs with Vapi Squads
Track escalation metrics: rate, time to escalation, post-escalation satisfaction

The Problem

Voice agents have limits. They can't handle angry callers, complex edge cases, payment disputes, or situations requiring human judgment. Without a clear escalation path, the agent loops, frustrates the caller, or makes promises it can't keep. With proper escalation, the caller feels taken care of even when the agent can't help.

The Concept

When to Escalate

Trigger	Threshold	Action
Sentiment: frustrated/angry	2+ negative turns	Transfer to human
Same question repeated	3+ times	Transfer to human
Tool failure	2+ retries failed	Transfer to human
Out of scope request	Agent can't handle	Transfer to human
Payment/refund >$500	Amount threshold	Transfer to human
Legal/compliance issue	Topic detection	Transfer to human
Caller asks for human	Direct request	Transfer immediately
Call duration >10 min	Time threshold	Offer transfer

A caller has asked "What's my refund status?" three times and the agent's tool keeps returning an error. What should happen?

Tell the caller to call back later

Escalation Types

Type	Description	Example
Cold transfer	Transfer without context	"Transferring you now."
Warm transfer	Transfer with context summary	"I'm transferring you to Sarah. She knows about your Tokyo flight issue."
Multi-agent handoff	Transfer to another AI agent	Billing agent → support agent
Scheduled callback	Schedule human callback	"A specialist will call you within 2 hours."
Co-browsing	Human joins the call	Human listens and assists

Warm Transfer with Context

class WarmTransfer:
    async def transfer(self, caller_id, context_summary, reason):
        """Transfer to human with context."""
        # Generate context summary for human agent
        summary = await self._generate_summary(context_summary)

        # Find available human agent
        human = await self._find_available_agent(skill="refunds")

        # Send context to human's dashboard
        await self._send_context(human.id, {
            "caller_id": caller_id,
            "summary": summary,
            "reason": reason,
            "sentiment": "frustrated",
            "call_duration": "4m 32s",
        })

        # Inform caller
        return f"I'm connecting you with {human.name}. They'll have all the details of our conversation."

Multi-Agent Handoff

For complex systems, multiple AI agents handle different domains:

class MultiAgentRouter:
    def __init__(self):
        self.agents = {
            "booking": BookingAgent(),
            "billing": BillingAgent(),
            "support": SupportAgent(),
            "general": GeneralAgent(),
        }

    async def route(self, intent, context):
        """Route to the appropriate specialist agent."""
        agent = self.agents.get(intent, self.agents["general"])

        # Transfer context to new agent
        await agent.receive_context(context)

        return agent

Escalation Flow

Context Summary for Handoff

async def generate_handoff_summary(conversation_history):
    """Generate a concise summary for the human agent."""
    prompt = f"""Summarize this call for a human agent in 2-3 sentences:
- What the caller wants
- What's been done so far
- Why escalation is needed
- Caller sentiment

Conversation:
{conversation_history}

Summary:"""

    return await llm.generate(prompt, max_tokens=80)

Escalation Message Templates

Situation	Agent Says
Frustration	"I can see this is frustrating. Let me connect you with a specialist who can help."
Tool failure	"I'm having trouble accessing that information. Let me transfer you to someone who can assist."
Out of scope	"That's outside what I can help with, but I can connect you with the right team."
Payment dispute	"For payment issues, I'll transfer you to our billing specialist."
Direct request	"Of course, let me connect you with a human agent right away."
Complex issue	"This needs a human touch. Let me find the right person for you."

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Agent