Phase 3: Transformers for NLP · ~60 minutes · Python

Chunking Strategies for RAG

Chunking configuration influences retrieval quality as much as the choice of embedding model (Vectara NAACL 2025). Get chunking wrong and no amount of reranking saves you.

Hiring signal: Understanding of chunking strategies for rag internals

What you will learn

Implement chunking strategies for rag from scratch
Understand the math and intuition behind the algorithm
Use production libraries for the same task
Ship a reusable artifact

Introduction

Type: Build Languages: Python Prerequisites: Phase 5 · 14 (Information Retrieval), Phase 5 · 22 (Embedding Models) Time: ~60 minutes

The Problem

You put a 50-page contract into a RAG system. User asks: "What is the termination clause?" The retriever returns the cover page. Why? Because the model was trained on 512-token chunks and the termination clause sits 20 pages in, split across a page break, with no local keywords tying it to the query.

The fix is not "buy a better embedding model." The fix is chunking. How big? Overlap? Where to split? With surrounding context?

Feb 2026 benchmarks show surprising results:

Vectara's 2026 study: recursive 512-token chunking beat semantic chunking 69% → 54% accuracy.
SPLADE + Mistral-8B on Natural Questions: overlap provided zero measurable benefit.
Context cliff: response quality drops sharply around 2,500 tokens of context.

The "obvious" answer (semantic chunking, 20% overlap, 1000 tokens) is often wrong. This lesson builds intuition for six strategies and tells you when to reach for which.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Concept, Build It, Pitfalls, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.