Phase 6: Automatic Speech Recognition · ~75 minutes · Python
Text-to-Speech (TTS) — From Tacotron to F5 and Kokoro
ASR inverts speech to text; TTS inverts text to speech. The 2026 stack is three parts: text → tokens, tokens → mel, mel → waveform. Each part has a default model that fits in a laptop.
What you will learn
- Implement text-to-speech (tts) — from tacotron to f5 and kokoro from scratch
- Understand the math and intuition behind the algorithm
- Use production libraries for the same task
- Ship a reusable artifact
Introduction
- Type: Build
- Languages: Python
- Prerequisites: Spectrograms, Mel Scale & Audio Features, Sequence-to-Sequence Models, The Full Transformer — Encoder + Decoder (Transformers & LLMs from Scratch)
- Time: ~75 minutes
The Problem
You have a string: "Please remind me to water the plants at 6 pm." You need a 3-second audio clip that sounds natural, has correct prosody (pauses, stress), pronounces "plants" with the right vowel, and runs in under 300 ms on a CPU for a live voice assistant. You also need to swap voices, handle code-switched input ("remind me at 6 pm, daijoubu?"), and not embarrass yourself on names.
Modern TTS pipelines look like this:
- Text frontend. Normalize text (dates, numbers, emails), convert to phonemes or subword tokens, predict prosody features.
- Acoustic model. Text → mel spectrogram. Tacotron 2 (2017), FastSpeech 2 (2020), VITS (2021), F5-TTS (2024), Kokoro (2024).
- Vocoder. Mel → waveform. WaveNet (2016), WaveRNN, HiFi-GAN (2020), BigVGAN (2022), neural codec vocoders in 2024+.
In 2026 the acoustic + vocoder split blurs with end-to-end diffusion and flow-matching models. But the mental model of three parts still holds for debugging.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers The Concept, Build, Check Yourself, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.
Create a free account — the opening phases of 29 of 35 courses are free, no credit card.
Browse all courses · View pricing · DeVenture Academy