The Problem
A user types: "Show me the critical unresolved support tickets about login issues on iOS filed in the last two weeks." Your semantic search embeds this and returns the 200 closest matches by cosine similarity to the word "login issues." Some are resolved. Many are on Android. Several are from 2022. A handful are feature requests, not bugs. The user wanted 8 tickets; they got 200.
The problem is that semantic similarity cannot enforce hard constraints. Cosine similarity measures conceptual proximity — "iOS login crash" is semantically close to "Android login crash" — but your user wants iOS only, and an Android result is not an acceptable substitute regardless of how semantically similar it is.
Self-querying retrieval solves this by having the LLM translate the natural language query into a structured query object: a semantic search string for the conceptual part ("login issues") plus metadata filters for the constraint parts (severity = "critical", status = "unresolved", platform = "iOS", date > 2 weeks ago). The retriever applies semantic search AND metadata filtering simultaneously, returning only results that are both semantically relevant AND satisfy every hard constraint.
This is the feature that makes enterprise search actually deployable. A legal team asking for "high-risk vendor contracts from APAC signed before 2023" is not asking for documents that are vaguely contract-like — they're specifying hard constraints that must hold. A security team asking for "critical unpatched CVEs in our authentication components" cannot afford to get medium-severity results mixed in. Self-querying handles these cases where pure semantic search fails.
Metadata schema design is the real work
Self-querying is only as powerful as your document metadata. An ideal schema covers every dimension a user might want to filter on: dates, categorical fields, severity levels, status flags, regions, versions. Retrofitting metadata onto a corpus after the fact is painful — design the schema before ingestion, and include typed fields with controlled vocabularies (enums) wherever possible.
Designing document metadata for self-querying
The metadata schema is the foundation. For a support ticket corpus:
TICKET_METADATA_SCHEMA = {
"ticket_id": "string", # unique identifier
"date_filed": "ISO 8601 date", # filter: date_after, date_before
"category": Literal[ # filter: eq, in
"technical", "billing", "shipping", "account"
],
"subcategory": "string", # filter: eq
"platform": Literal[ # filter: eq, in
"iOS", "Android", "web", "api"
],
"severity": Literal[ # filter: eq, gte
"low", "medium", "high", "critical"
],
"status": Literal[ # filter: eq, in
"open", "in_progress", "resolved", "closed"
],
"product_version": "string" # filter: eq, gte
}
Controlled vocabularies (Literals) are important: they let the LLM generate exactly-valid filter values rather than free-text strings that might not match your database. When defining the schema in your self-querying prompt, include the allowed values for each enum field explicitly.
The difference between good and bad metadata shows up in a concrete test. Query: "recent high-severity billing issues." With rich metadata: {semantic_query: "billing issues", filters: {category: "billing", severity: "high", date_after: "30_days_ago"}, sort: "date_desc"} → 12 precise results. With just document text and no metadata: semantic search on "billing issues" → 180 results, all relevance-ranked, severity mixed throughout.
Implementing self-querying retrieval
The self-querying retriever has two stages. First, an LLM call that generates the structured query:
structured_query_prompt = f"""
You are a query translator. Convert the user's natural language query into a structured query.
Available metadata filters:
{json.dumps(TICKET_METADATA_SCHEMA, indent=2)}
For category and platform, only use values from the allowed list.
For dates, use ISO 8601 format.
User query: "{user_query}"
Return JSON:
{{
"semantic_query": "the conceptual part of the query for vector search",
"filters": {{...}},
"sort": "date_desc | severity_desc | score_desc",
"top_k": 5
}}
"""
Second, the retriever executes the structured query against ChromaDB using the where parameter for metadata filters:
results = collection.query(
query_texts=[structured_query.semantic_query],
where=build_chroma_filter(structured_query.filters),
n_results=structured_query.top_k
)
ChromaDB's where syntax supports $eq, $ne, $gt, $gte, $lt, $lte, $in, and $and/$or for compound filters — map your structured query filters to these operators.
A user asks "Find the most critical unresolved bugs filed in the last 2 weeks." A self-querying retriever should produce:
Self-querying separates the semantic concept ("bugs" — the thing you're searching for) from the hard constraints (severity must be critical, status must be unresolved, date must be recent). The semantic search finds conceptually relevant items; the metadata filters apply non-negotiable constraints. Putting every condition into one semantic query collapses this distinction — "critical unresolved bugs last 2 weeks" as a single embedding will retrieve documents that are semantically close to all those words but don't necessarily satisfy any one of them as a hard filter.
Recursive retrieval — following the graph
Documents reference each other. A top-level policy document says "see also: Section 4.2 of the Procurement Guide." A code file imports three modules. A legal contract incorporates by reference five exhibits. If you only retrieve the top document, you miss the context that the references point to.
Recursive retrieval follows these references: retrieve the initial result set, parse each document for reference links, retrieve those documents too, up to a configurable depth. The algorithm:
def recursive_retrieve(
query: str,
collection,
max_depth: int = 2,
visited: set | None = None
) -> list[Document]:
if visited is None:
visited = set()
# Initial retrieval
results = collection.query(query_texts=[query], n_results=5)
docs = parse_results(results)
expanded = []
for doc in docs:
if doc.id in visited:
continue
visited.add(doc.id)
expanded.append(doc)
if max_depth > 0 and doc.metadata.get("references"):
for ref_id in doc.metadata["references"]:
if ref_id not in visited:
ref_docs = collection.get(ids=[ref_id])
expanded.extend(
recursive_retrieve(
query=query,
collection=collection,
max_depth=max_depth - 1,
visited=visited
)
)
return expanded
Two critical implementation details:
- Cycle detection via
visited set: without tracking visited document IDs, circular references (Doc A → Doc B → Doc A) cause infinite recursion. The visited set is passed through all recursive calls to prevent re-processing any document.
- Depth cap: practical maximum is 2–3 levels. Beyond depth 2, retrieved documents are typically tangentially related to the original query — the signal-to-noise ratio degrades. At depth 3+, you're often retrieving the entire corpus through transitive references.
Recursive retrieval should always be capped at a maximum depth because:
The cycle problem is the critical safety reason — without a depth cap and the visited-set pattern, one circular reference in your document graph causes unbounded recursion. Depth D also addresses the noise concern (option 3 is a real consideration), but it's secondary to preventing infinite loops. Both a depth limit AND a visited-set are required: the depth limit alone doesn't protect against cycles within the same depth level.
Build
Build it
Here's a minimal self-querying filter matcher — no vector DB, just the logic of applying metadata constraints alongside a keyword-based "semantic" score.
tickets = [
{"id": "T1", "text": "login crash on iOS app", "platform": "iOS", "severity": "critical", "status": "unresolved"},
{"id": "T2", "text": "login crash on Android app", "platform": "Android", "severity": "critical", "status": "unresolved"},
{"id": "T3", "text": "iOS login slow to respond", "platform": "iOS", "severity": "low", "status": "resolved"},
{"id": "T4", "text": "iOS app login fails after update", "platform": "iOS", "severity": "critical", "status": "unresolved"},
]
def matches_filters(ticket: dict, filters: dict) -> bool:
return all(ticket.get(key) == value for key, value in filters.items())
def semantic_score(text: str, query_words: set) -> float:
words = set(text.lower().split())
return len(words & query_words) / len(query_words)
def self_query(semantic_query: str, filters: dict, top_k: int = 5):
query_words = set(semantic_query.lower().split())
candidates = [t for t in tickets if matches_filters(t, filters)]
scored = [(semantic_score(t["text"], query_words), t) for t in candidates]
scored.sort(key=lambda pair: pair[0], reverse=True)
return scored[:top_k]
results = self_query("login issues", {"platform": "iOS", "severity": "critical", "status": "unresolved"})
for score, ticket in results:
print(f"{ticket['id']}: score={score:.2f} -- {ticket['text']}")
Build lesson_03_03_self_querying.py yourself — this is the lab. A complete implementation covers both retrieval patterns with comparison benchmarking and contains:
DocumentMetadataSchema: define typed metadata fields with allowed values for ChromaDB ingestionSelfQueryingRetriever: LLM call to generate {semantic_query, filters, sort, top_k} from natural language, then execute against ChromaDB using where filter syntaxRecursiveRetriever: follow document references metadata up to configurable max_depth, with visited-set cycle detection- Demo corpus: 50 support tickets with rich metadata (category, platform, severity, status, date) and inter-ticket reference links
- Comparison benchmark: pure semantic search vs. self-querying on 10 filtered queries — shows Precision@5 improvement