Phase 7: Speech Synthesis & Applications · ~45 minutes · Python

Voice Activity Detection & Turn-Taking — Silero, Cobra, and the Flush Trick

Every voice agent lives or dies on two decisions: is the user speaking now, and are they done? VAD answers the first. Turn-detection (VAD + silence-hangover + semantic endpoint model) answers the second. Get either wrong and your assistant either cuts users off or never shuts up.

What you will learn

Implement voice activity detection & turn-taking — silero, cobra, and the flush trick from scratch
Understand the math and intuition behind the algorithm
Use production libraries for the same task
Ship a reusable artifact

Introduction

Type: Build
Languages: Python
Prerequisites: Real-Time Audio Processing, Build a Voice Assistant Pipeline (Capstone)
Time: ~45 minutes

The Problem

Three distinct decisions a voice agent makes on every 20 ms chunk:

Is this frame speech? — VAD. Binary, per-frame.
Has the user started a new utterance? — onset detection.
Has the user finished? — end-pointing (turn-end).

The naive answer (energy threshold) fails on any noise — traffic, keyboards, crowd babble. The 2026 answer: Silero VAD (open, deep-learned) + a turn-detection model (semantic endpointing) + a VAD-calibrated silence hangover.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Concept, Build, Check Yourself, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.