The Problem
"How did the 2021 policy change in our procurement department affect the vendor onboarding process, given the contracts we signed with our logistics partners in 2019?" This question cannot be answered from any single document chunk. The relationship between the 2021 policy, the onboarding process, and the 2019 contracts exists across five documents, each of which contains one piece of the picture. Standard RAG retrieves the five most semantically similar chunks — but "semantically similar" is the wrong criterion here. What you need is relational traversal: find the entities, find how they connect.
GraphRAG, published by Microsoft Research in April 2024, addresses exactly this gap. The insight is that a corpus of documents is not just a bag of text — it's a latent knowledge graph waiting to be made explicit. Every document mentions entities (people, organizations, concepts, policies, products) and the relationships between them ("the procurement policy governs vendor contracts," "vendor contracts were amended in 2021," "the GDPR team reviews all vendor contracts"). GraphRAG extracts this graph during indexing, enabling query-time reasoning that traverses entity relationships across document boundaries.
This is not a niche research contribution — Microsoft open-sourced the library in 2024 and it has since been adopted by enterprise knowledge management teams at large organizations. The pattern of building knowledge graphs from unstructured documents for downstream reasoning is now standard architecture knowledge for AI engineers working on complex enterprise search.
Global vs. Local Search
GraphRAG supports two retrieval modes that address different question types. Local search combines retrieved entity/relationship data with the original text chunks — best for specific entity questions ("What is the relationship between X and Y?"). Global search searches across community summaries — best for thematic questions about the entire corpus ("What are the main themes across all our documents?"). Standard vector RAG cannot do global search at all — it has no concept of the whole corpus.
The GraphRAG architecture
Microsoft's GraphRAG pipeline has five phases. Understanding each phase is necessary to implement it and to debug it when it produces unexpected results.
Phase 1 — Entity extraction: For each document chunk, an LLM extracts a list of named entities and the relationships between them. Prompt: "Extract all entities (people, organizations, policies, concepts, products, locations) and the relationships between them. Return JSON: [{entity_1, entity_type, entity_2, relationship, description}]." This runs on every chunk — the most expensive phase, consuming most of the indexing token budget.
Phase 2 — Graph construction: Merge entity mentions across chunks into a property graph. Entity deduplication: "procurement policy" and "the 2021 procurement policy" are the same node. Build the graph: nodes = unique entities, edges = relationships extracted across all chunks, edge weight = frequency of the relationship appearing in the corpus.
Phase 3 — Community detection: Run the Leiden algorithm (a variant of Louvain with better modularity) on the entity graph to find clusters of strongly interconnected entities. A community might be: {GDPR compliance team, vendor contracts, legal review, onboarding process} — a cluster of entities that frequently appear together. Community detection is done once at indexing time.
Phase 4 — Community summaries: For each community detected, run an LLM summarization over all the text chunks containing entities in that community. This produces a dense paragraph that captures what the community is "about." These summaries are stored and indexed for global search.
Phase 5 — Retrieval: At query time, decide: local search (retrieve entities + relationships + original chunks) or global search (retrieve relevant community summaries). Combine and send to the LLM for final synthesis.
A user asks: "What is the relationship between the GDPR compliance team and the vendor onboarding process, based on our internal documents?" Standard RAG returns 5 chunks that each mention one or the other but not their connection. What does GraphRAG do differently?
This is GraphRAG's core advantage. The relationship between "GDPR compliance team" and "vendor onboarding" might be scattered across 6 documents — no single chunk says "GDPR team reviews all new vendor contracts before onboarding." GraphRAG extracted entity-relationship pairs from each chunk at indexing time, building graph edges like: GDPR team --reviews--> vendor contracts --required_for--> onboarding process. At query time, it traverses these edges to find the connection, rather than hoping semantic similarity retrieves the right chunks.
Implementing a simplified GraphRAG pipeline
The full Microsoft GraphRAG library handles production-scale complexity. For understanding the architecture, a simplified implementation using the Anthropic API for entity extraction and NetworkX for graph construction is illuminating:
import networkx as nx
import json
# Phase 1: Entity extraction per chunk
def extract_entities_and_relations(chunk: str, client) -> list[dict]:
response = client.messages.create(
model="claude-opus-4-5",
max_tokens=1024,
messages=[{
"role": "user",
"content": f"""Extract entities and relationships from this text.
Return JSON array: [{{"entity_1": str, "type_1": str, "entity_2": str, "type_2": str, "relationship": str}}]
Text: {chunk}"""
}]
)
return json.loads(response.content[0].text)
# Phase 2: Build NetworkX graph
def build_graph(all_extractions: list[list[dict]]) -> nx.Graph:
G = nx.Graph()
for extractions in all_extractions:
for item in extractions:
G.add_node(item["entity_1"], type=item["type_1"])
G.add_node(item["entity_2"], type=item["type_2"])
if G.has_edge(item["entity_1"], item["entity_2"]):
G[item["entity_1"]][item["entity_2"]]["weight"] += 1
G[item["entity_1"]][item["entity_2"]]["relations"].append(item["relationship"])
else:
G.add_edge(
item["entity_1"], item["entity_2"],
weight=1,
relations=[item["relationship"]]
)
return G
# Phase 3: Community detection (Leiden via python-louvain as approximation)
from community import best_partition
def detect_communities(G: nx.Graph) -> dict[str, int]:
return best_partition(G) # {entity: community_id}
Local vs. global search
Local search is the right mode when the user is asking about specific entities or relationships:
- "How is entity X related to entity Y?"
- "What do we know about [specific organization]?"
- "What decisions were made regarding [specific policy]?"
Local search retrieves: the relevant entity nodes and their direct edges, the original text chunks that mention those entities, and the community summary for the communities those entities belong to. Combine all three and pass to the LLM.
Global search is the right mode when the user is asking about the corpus as a whole:
- "What are the main themes across all our customer complaint documents?"
- "How has our procurement policy evolved over time?"
- "What are the key risk areas mentioned across our compliance documents?"
Global search retrieves community summaries ranked by relevance to the query. Standard vector RAG cannot do this — it has no mechanism to represent "what is the document corpus as a whole about." GraphRAG's community summaries are pre-computed answers to exactly that kind of question.
You have a 10,000-document corpus. Which question is GraphRAG dramatically better at than standard RAG?
Question B requires global reasoning — understanding themes across all 10,000 documents, tracking their evolution. Standard RAG retrieves the top-k most semantically similar chunks. It has no concept of "the entire corpus" and cannot reason about what the full body of complaints is "about." GraphRAG's community summaries are pre-computed syntheses of the entity clusters found across the full corpus. Global search retrieves these summaries, giving the LLM a compressed representation of the whole knowledge base. Questions A, C, and D are simple lookups that standard RAG handles well — GraphRAG's extra complexity is not justified for them.
When to use GraphRAG vs. standard RAG
| Factor | Use Standard RAG | Use GraphRAG |
|---|
| Question type | Factual lookups, specific document retrieval | Multi-hop relationship queries, global thematic analysis |
| Corpus size | Any | Meaningful at >500 docs; justifies cost at >2,000 |
| Indexing budget | Limited | Can afford 100k–500k tokens per 1,000 docs |
| Latency requirement | <500ms | Can tolerate 1–3s for complex queries |
| Relationship density | Low — docs are independent | High — docs reference each other, share entities |
The cost realism matters: indexing 1,000 documents through GraphRAG's entity extraction phase consumes roughly 100,000–500,000 tokens depending on chunk size and extraction prompt complexity. At claude-opus-4-5 pricing, this is a real indexing cost that must be justified by the query types. If your users are mostly asking simple factual questions, standard RAG wins.