Phase 5: Turn-Taking & Conversation Management · 55 min · Python · Pipecat · LiveKit Agents SDK

Interruption Handling (Barge-In)

Interruption handling is the #1 differentiator between good and bad voice agents. Get it right or users hate your agent.

Hiring signal: Barge-in implementation is the most critical differentiator tested in voice agent evaluation.

What you will learn

Detect barge-in: user speech during agent TTS playback
Execute the interruption response: stop TTS, cancel LLM, clear buffer, restart loop
Implement interruption handling in Pipecat and LiveKit
Measure and minimize false interruption rate

The Problem

The agent is speaking: "I can help you book a flight to Tokyo. What date would you like to—" and the caller interrupts: "Actually, I need to go to Osaka instead." If the agent doesn't detect this interruption (barge-in), it continues talking over the caller, creating a frustrating experience.

The Concept

What is Barge-In?

Barge-in is when a caller speaks while the agent is talking. The system must:

Detect the interruption (VAD during TTS playback)
Stop TTS immediately (cut off the agent's audio)
Cancel the LLM stream (stop generating remaining tokens)
Start ASR (capture what the caller is saying)
Process the new input (respond to the interruption)

When barge-in is detected, what must happen in what order?

Start ASR → Stop TTS → Process input → Cancel LLM

Barge-In Detection

class BargeInDetector:
    def __init__(self, vad, agent_speaking=False):
        self.vad = vad
        self.agent_speaking = agent_speaking
        self.barge_in_count = 0

    def check(self, caller_audio, current_time):
        """Check if caller is interrupting while agent speaks."""
        if not self.agent_speaking:
            return False

        # Use VAD to detect caller speech
        is_speech = self.vad.is_speech(caller_audio, 16000)

        if is_speech:
            self.barge_in_count += 1
            return True

        return False

Echo Cancellation Challenge

When the agent is speaking through a speaker, the microphone may pick up the agent's own TTS audio. This is acoustic echo — the system must distinguish the agent's voice from the caller's voice.

Solution	How	Complexity
Simple VAD threshold	Higher threshold during TTS	Low (but unreliable)
Echo cancellation	DSP filter removes TTS audio from mic	High
Full-duplex AEC	Hardware echo cancellation	Very High
TTS-aware VAD	Compare mic audio to TTS output	Medium

Handling Partial Responses

When barge-in occurs, the LLM may have already generated part of a response. You need to:

Keep what was already spoken (don't repeat it)
Discard what wasn't spoken (no longer relevant)
Generate a fresh response based on the interruption

async def handle_barge_in(self, partial_response, caller_input, context):
    """Handle barge-in with context preservation."""
    # What was already spoken
    spoken_text = partial_response[:self.spoken_char_count]

    # Add to context so LLM knows what was said
    context.append({
        "role": "assistant",
        "content": spoken_text + " [interrupted]"
    })
    context.append({
        "role": "user",
        "content": caller_input
    })

    # Generate fresh response
    new_response = await self.llm.generate(context)
    return new_response

Barge-In Modes

Mode	Behavior	Use Case
Full barge-in	Stop TTS, cancel LLM, process new input	Default
Partial barge-in	Stop TTS but keep LLM response	Agent finishes in text
No barge-in	Agent continues, queue caller input	Formal announcements
Cooperative	Pause TTS, resume after caller finishes	Backchanneling

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Agent