Phase 3: Agentic RAG · 65 min · Python · Anthropic SDK · ChromaDB

Multi-Source Retrieval

Enterprise knowledge lives in five places at once — federated retrieval queries them all in parallel and returns a single ranked answer.

Hiring signal: Enterprise AI engineers are consistently tasked with connecting AI to existing data infrastructure — not building green-field systems. Multi-source retrieval is the practical skill that answers 'how would you build a knowledge assistant that works across our existing tools?' — the question asked at every enterprise AI interview.

What you will learn

Build a federated retrieval system that queries multiple sources in parallel using asyncio
Implement Reciprocal Rank Fusion (RRF) to merge results from sources with incompatible score scales
Design source connectors with a standard interface for easy extension to new data sources

The Problem

The enterprise knowledge problem isn't finding documents — it's that documents live everywhere and moving them all into one place is rarely an option. A user question about a recent customer complaint might require: the original ticket from your support system, the order history from your SQL database, the product page from your wiki, and a recent Slack thread where your team discussed a workaround. None of these live in the same place. None use the same retrieval API.

The naive approach is to pick one source and accept that the answer will be incomplete. The better approach is federated retrieval: define a standard retrieval interface, implement a connector for each source, query them all in parallel, merge and re-rank the results, and return the top-K across all sources as if they were a single corpus.

This pattern is not theoretical. Every enterprise AI assistant that achieves broad knowledge coverage uses some version of it. Salesforce Einstein, Workday Assistant, ServiceNow's AI features — they all federate across CRM data, HR systems, ticketing systems, and documentation. The engineering challenge is merging results from sources that use completely different relevance scoring schemes.

Reciprocal Rank Fusion (RRF)
RRF is the standard algorithm for merging ranked lists from sources with incompatible score scales. A vector DB returns cosine similarity (0–1). A full-text search returns BM25 scores (0–∞). A recency API might score by inverse age. These numbers cannot be averaged directly. RRF ignores raw scores entirely — it only uses each document's rank position within each source's list, making it distribution-agnostic. Formula: score(d) = Σ 1/(k + rank(d)) across all sources.

The Retriever Interface

The plugin architecture starts with a single abstract interface all connectors implement:

from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any

@dataclass
class Document:
    id: str
    text: str
    source: str
    score: float
    metadata: dict[str, Any]

class Retriever(ABC):
    @abstractmethod
    async def search(
        self,
        query: str,
        filters: dict | None = None,
        top_k: int = 10
    ) -> list[Document]:
        """Return top_k documents ranked by relevance to query."""
        ...

Every connector — vector DB, SQL, web search, Slack, Confluence — implements this interface. The FederatedRetriever doesn't know or care what's behind each connector; it calls search() on all of them. Adding a new source means writing one new Retriever subclass, not changing the agent or the federated retriever.

Three real connector patterns:

VectorDBRetriever (ChromaDB): Embed the query, run collection.query(), return results with cosine similarity scores. Score range: 0–1.

SQLRetriever (PostgreSQL full-text): Use to_tsvector and to_tsquery for full-text search, or use LIKE/ILIKE for simpler cases. Return rows ranked by ts_rank. Score range: 0–1 from ts_rank, or 0–∞ from raw BM25.

WebSearchRetriever: Call a search API (Brave, Bing, Tavily), return results ranked by the API's relevance score. Score range: proprietary.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Reciprocal Rank Fusion, Cross-Source Deduplication, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

RAG (Retrieval-Augmented Generation)