Phase 6: Dify & Visual AI App Builders · 50 min · Dify · Weaviate · OpenAI API

Dify Knowledge Base Deep Dive — Chunking, Indexing, and Retrieval

Dify's built-in knowledge base is the fastest path from documents to RAG.

Hiring signal: Dify's knowledge base management is its killer feature for RAG. Being able to configure chunking modes (General, Parent-child, Q&A), index methods (High Quality vs Economical), and retrieval settings (top-k, score threshold, reranking) demonstrates hands-on Dify RAG experience that AI solutions roles require.

What you will learn

Configure Dify knowledge base chunking modes: General, Parent-child, Q&A
Choose index methods: High Quality (vector/full-text/hybrid) vs Economical (inverted index)
Configure retrieval settings: top-k, score threshold, reranking model
Connect knowledge bases to Dify workflows and chatbot apps

The Problem

You've used Dify's knowledge base at a basic level — upload documents, connect to a chatbot, it answers questions. But the answers are sometimes wrong or generic. The retrieval isn't finding the right chunks. You need to understand the configuration options: chunking modes, index methods, retrieval settings, and reranking — and how each affects retrieval quality.

Dify's knowledge base is powerful but the defaults aren't always optimal. Understanding the configuration options is the difference between "it kind of works" and "it retrieves the right information every time."

Dify's knowledge base config determines RAG quality
Three configuration layers affect retrieval: (1) chunking mode — how documents are split, (2) index method — how chunks are stored and searched, (3) retrieval settings — how many chunks, what threshold, whether to rerank. Getting all three right is the difference between 70% and 90% retrieval accuracy.

The Concept

Dify Knowledge Base Configuration Layers

┌──────────────────────────────────────────────────────────────┐
│  DIFY KNOWLEDGE BASE CONFIGURATION                            │
│                                                              │
│  Layer 1: CHUNKING MODE                                      │
│  ├── General: Fixed-size chunks with overlap                 │
│  ├── Parent-Child: Small chunks match, parent provides ctx   │
│  └── Q&A: LLM generates Q&A pairs as chunks                  │
│                                                              │
│  Layer 2: INDEX METHOD                                       │
│  ├── High Quality (recommended)                              │
│  │   ├── Vector: semantic search via embeddings              │
│  │   ├── Full-text: keyword search (BM25-style)              │
│  │   └── Hybrid: combine vector + full-text                  │
│  └── Economical: inverted index only (no embeddings)         │
│                                                              │
│  Layer 3: RETRIEVAL SETTINGS                                 │
│  ├── Top-K: number of chunks to retrieve (3–10)              │
│  ├── Score threshold: minimum similarity (0.0–1.0)           │
│  └── Reranking: Cohere/Jina reranker (optional)              │
└──────────────────────────────────────────────────────────────┘

Chunking Mode Comparison

Mode	How It Works	Best For	Cost
General	Fixed-size chunks (e.g., 500 tokens) with overlap	Most documents (default)	Low
Parent-Child	Small chunks (256) for matching, parent (1024) for context	Long documents needing context	Medium
Q&A	LLM generates Q&A pairs from document	FAQs, support docs, structured Q&A	High (LLM processing)

GENERAL CHUNKING:
  Doc: "Return policy: 30 days unopened. 14 days opened with 20% restocking fee."
  → Chunk 1: "Return policy: 30 days unopened."
  → Chunk 2: "14 days opened with 20% restocking fee."
  Problem: Chunk 2 doesn't mention "return policy" — may not match "What's the return policy?"

PARENT-CHILD CHUNKING:
  Parent: "Return policy: 30 days unopened. 14 days opened with 20% restocking fee."
  Child 1: "30 days unopened"
  Child 2: "14 days opened with 20% restocking fee"
  Search: match on Child 2 ("restocking fee")
  Return: Parent (full context including "Return policy")

Q&A CHUNKING:
  Q: "What is the return policy for unopened items?" A: "30 days"
  Q: "What is the return policy for opened items?" A: "14 days with 20% restocking fee"
  Search: match on question → return Q&A pair
  Best for: direct question matching

You're building a RAG system for a 500-page product manual. Users ask specific questions like "What's the torque spec for bolt A-12?" Which chunking mode is best?

Parent-Child is best for a large technical manual. Users ask specific questions ("torque spec for bolt A-12") that match small chunks (child), but the answer needs surrounding context (parent) — like the section on engine assembly. General chunking might split the torque spec from its context. Q&A chunking would require the LLM to generate questions for every section of a 500-page manual — expensive and may miss specific technical details.

Index Method Comparison

Method	How It Works	Accuracy	Cost	Best For
Vector	Embedding similarity search	Good for conceptual matches	Embedding API cost	Most use cases
Full-text	BM25 keyword search	Good for exact terms	No embedding cost	Specific terms, codes, IDs
Hybrid	Vector + full-text combined	Best overall accuracy	Embedding + search cost	Production (recommended)
Economical	Inverted index only	Lower accuracy	Free	Testing, budget-constrained

Retrieval Settings

Setting	Default	Range	Effect
Top-K	3	1–10	More chunks = more context but more noise
Score threshold	0.5	0.0–1.0	Higher = fewer but more relevant results
Reranking	Off	On/Off	Re-orders candidates by true relevance

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Exercises, Key Terms, Common Pitfalls — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.