The Problem
The enterprise knowledge problem isn't finding documents — it's that documents live everywhere and moving them all into one place is rarely an option. A user question about a recent customer complaint might require: the original ticket from your support system, the order history from your SQL database, the product page from your wiki, and a recent Slack thread where your team discussed a workaround. None of these live in the same place. None use the same retrieval API.
The naive approach is to pick one source and accept that the answer will be incomplete. The better approach is federated retrieval: define a standard retrieval interface, implement a connector for each source, query them all in parallel, merge and re-rank the results, and return the top-K across all sources as if they were a single corpus.
This pattern is not theoretical. Every enterprise AI assistant that achieves broad knowledge coverage uses some version of it. Salesforce Einstein, Workday Assistant, ServiceNow's AI features — they all federate across CRM data, HR systems, ticketing systems, and documentation. The engineering challenge is merging results from sources that use completely different relevance scoring schemes.
Reciprocal Rank Fusion (RRF)
RRF is the standard algorithm for merging ranked lists from sources with incompatible score scales. A vector DB returns cosine similarity (0–1). A full-text search returns BM25 scores (0–∞). A recency API might score by inverse age. These numbers cannot be averaged directly. RRF ignores raw scores entirely — it only uses each document's rank position within each source's list, making it distribution-agnostic. Formula: score(d) = Σ 1/(k + rank(d)) across all sources.
The retriever interface
The plugin architecture starts with a single abstract interface all connectors implement:
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any
@dataclass
class Document:
id: str
text: str
source: str
score: float
metadata: dict[str, Any]
class Retriever(ABC):
@abstractmethod
async def search(
self,
query: str,
filters: dict | None = None,
top_k: int = 10
) -> list[Document]:
"""Return top_k documents ranked by relevance to query."""
...
Every connector — vector DB, SQL, web search, Slack, Confluence — implements this interface. The FederatedRetriever doesn't know or care what's behind each connector; it calls search() on all of them. Adding a new source means writing one new Retriever subclass, not changing the agent or the federated retriever.
Three real connector patterns:
VectorDBRetriever (ChromaDB): Embed the query, run collection.query(), return results with cosine similarity scores. Score range: 0–1.
SQLRetriever (PostgreSQL full-text): Use to_tsvector and to_tsquery for full-text search, or use LIKE/ILIKE for simpler cases. Return rows ranked by ts_rank. Score range: 0–1 from ts_rank, or 0–∞ from raw BM25.
WebSearchRetriever: Call a search API (Brave, Bing, Tavily), return results ranked by the API's relevance score. Score range: proprietary.
Reciprocal rank fusion
RRF was introduced by Cormack, Clarke, and Buettcher in 2009 as a method to combine search results from heterogeneous retrieval systems without calibrating score distributions. The formula:
RRF_score(document d) = Σ_r∈retrievers 1 / (k + rank(d, r))
Where k is a smoothing constant (typically 60) and rank(d, r) is the 1-indexed position of document d in retriever r's ranked list. If a document doesn't appear in a source's list, it contributes 0 from that source.
def reciprocal_rank_fusion(
ranked_lists: dict[str, list[Document]],
k: int = 60,
source_weights: dict[str, float] | None = None
) -> list[Document]:
"""
Merge multiple ranked lists using RRF.
Args:
ranked_lists: {source_name: [Document, ...]} — already ranked by each source
k: smoothing constant (default 60, from original paper)
source_weights: optional per-source multiplier for trusted sources
"""
scores: dict[str, float] = {}
doc_registry: dict[str, Document] = {}
for source, docs in ranked_lists.items():
weight = source_weights.get(source, 1.0) if source_weights else 1.0
for rank, doc in enumerate(docs, start=1):
content_hash = hash(doc.text[:200])
scores[content_hash] = scores.get(content_hash, 0.0) + weight / (k + rank)
doc_registry[content_hash] = doc # last-write wins for metadata
return sorted(doc_registry.values(), key=lambda d: scores[hash(d.text[:200])], reverse=True)
The source_weights parameter lets you apply a trust multiplier — internal documentation from your own wiki might be weighted 1.5× vs. general web search results at 0.8×. This doesn't change the rank-based calculation; it scales each source's contribution before summing.
You federate across a vector DB (cosine similarity, 0–1), a full-text search engine (BM25, 0–∞), and a recency API (score = 1/days_since_published). How do you combine these rankings fairly?
RRF is designed for exactly this problem. Raw scores from these three sources are on completely incompatible scales — a BM25 score of 12.5 vs. a cosine score of 0.73 vs. a recency score of 0.033 cannot be meaningfully averaged. Normalizing scores to [0, 1] helps but requires knowing the distribution of each source's scores, which varies by query. RRF avoids this entirely by discarding raw scores and using only rank positions — a document ranked 1st in any source gets 1/(k+1) contribution from that source regardless of its raw score.
Cross-source deduplication
When multiple sources return the same document — for example, a product page that's both in your vector DB and returned by web search — including it twice wastes context tokens and can over-influence the model's answer by implying it's two independent sources agreeing.
Deduplication by content hash before RRF scoring:
def deduplicate_results(
ranked_lists: dict[str, list[Document]]
) -> dict[str, list[Document]]:
"""Remove documents seen in earlier-priority sources."""
seen_hashes: set[str] = set()
deduplicated: dict[str, list[Document]] = {}
for source, docs in ranked_lists.items():
unique_docs = []
for doc in docs:
content_hash = hashlib.md5(doc.text[:500].encode()).hexdigest()
if content_hash not in seen_hashes:
seen_hashes.add(content_hash)
unique_docs.append(doc)
deduplicated[source] = unique_docs
return deduplicated
For web results, URL deduplication is more reliable than content hash. For internal documents, use the document ID.
A document appears in both your vector DB results and your web search results for the same query. What should you do?
Duplicate results waste context window tokens and create an artificial consensus signal — the model sees the same information twice and may weigh it more heavily than warranted. Deduplication before RRF scoring ensures the document appears once with the combined relevance signal from both sources. The RRF formula naturally accounts for the cross-source agreement: a document ranked 3rd in source A and 2nd in source B gets 1/(k+3) + 1/(k+2) — a higher combined score than a document appearing in only one source.