GraphRAG and global context

Slide 1 of 20Open this slide

GraphRAG

When structure beats similarity

Week 6 — Applied AI Course

Knowledge graphs meet retrieval: index relationships, not just chunks.

Sources

• Edge et al., "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (Microsoft, 2024)
• Guo et al., "LightRAG: Simple and Fast Retrieval-Augmented Generation" (2024)
• microsoft/graphrag

Slide 2 of 20Open this slide

Where We Came From

Vector RAG retrieves text chunks whose embeddings are closest to the query. This works well when:

• The answer lives in a few specific passages
• The user's wording is close to the document's wording
• The corpus is large enough that semantic similarity is meaningful

What it ignores

A vector index treats the corpus as a bag of chunks. It throws away:

• Relationships between entities ("Acme Corp is owned by ConsolidatedCo") that span chunks
• The corpus structure — folder hierarchy, citation graph, version history, threaded conversation
• Aggregate properties that require reading many chunks together ("how many engineers left in Q3?")

For point lookups, this is fine. For questions that require connecting facts or summarizing a slice of the corpus, it falls down. That's where GraphRAG fits.

The benchmark numbers behind the hype

Microsoft's GraphRAG paper Edge et al. 2024 reports +20–80% comprehensiveness vs naive RAG on query-focused summarization tasks (using LLM-as-judge). LightRAG (Guo et al. 2024) shows similar gains at lower cost. Both compare against well-tuned RAG baselines, not strawmen.

Sources

• Lewis et al. — original RAG (2020)
• Edge et al. — GraphRAG paper
• Anthropic — Contextual Retrieval blog (2024)

Slide 3 of 20Open this slide

Three Question Shapes Vector RAG Can't Answer

Multi-hop questions

"Which of our customers run on AWS, and how many of them have open Tier-1 tickets?"

Vector RAG retrieves passages about customers, separately about AWS, separately about tickets. None of the top-k chunks contains the join. The model is left to fabricate it from partial information.

Global summarization

"What were the recurring themes in our customer interviews last quarter?"

The answer requires reading every interview transcript, then synthesizing. No top-k retrieval works; the question is global, not local.

Entity-centric queries with traversal

"What are all the products that depend on the auth service?"

The dependency graph spans many docs. A bag of chunks loses the graph structure. Even if every chunk is retrieved, the model has to reconstruct edges from prose.

What this looks like in eval

Naive RAG on a 1,000-document corpus, scored by LLM-judge on a benchmark of 100 questions split 80/20 between local and global:

(Numbers approximate, from Edge et al. 2024 §4.3. Local lookup parity is the expected result — GraphRAG isn't trying to win at point lookups.)

Implication

Vector RAG is not "wrong" — it's optimized for the local lookup shape. If your traffic is mostly local lookups, you don't need GraphRAG. If you have a mix, you need both, plus a router that picks the right one per query.

Sources

• Edge et al. — GraphRAG paper
• LangChain — RAG anti-patterns

Slide 4 of 20Open this slide

Index the Graph, Not Just the Chunks

GraphRAG augments — or replaces — the vector index with a structured knowledge graph. Entities become nodes; the relationships between them become edges. Retrieval can now traverse the graph, not just look up nearest neighbors in an embedding space.

What the LLM does at index time

A GraphRAG pipeline uses an LLM at indexing, not just at retrieval. For each chunk:

1. Extract entities — people, orgs, products, files, dates, anything domain-specific (a configurable taxonomy)
2. Extract relations — "Alice manages Bob", "Postgres is a dependency of OrderService", "Q3 launch postponed Q4 launch"
3. Generate descriptions — for each entity and relation, a short summary the system can search against later

This is expensive: many LLM calls during indexing. The payoff is that retrieval is cheap and structured — a graph traversal rather than a re-embedding cascade.

What the structure gives you

What's not new

The idea of indexing entities and relations predates LLMs by decades (ontology-based retrieval, semantic networks). What's new is that LLMs can do the extraction for free-form text — no schema, no NER training, no hand-coded rules.

Sources

• Edge et al. — GraphRAG paper
• microsoft/graphrag — reference implementation

Slide 5 of 20Open this slide

Turning Chunks Into Nodes

The first stage of the indexing pipeline. For each chunk, an LLM extracts named entities and their (optionally) typed attributes.

The extraction prompt (simplified)

EXTRACTION_PROMPT = """
Extract entities and their relationships from the text below.

Entity types to look for: {entity_types}

For each entity, output:
- name: the canonical name
- type: from the list above
- description: one sentence summarizing what the text says about this entity

For each relationship between two entities in this chunk, output:
- source, target: entity names
- description: how they are related (from the text)
- weight: 1-10 strength

Output JSON. Do not invent entities not mentioned in the text.

TEXT:
{chunk}
"""

A real pipeline (microsoft/graphrag) adds:

• A gleaning loop — after the first pass, ask the model "did you miss any?" up to N times to catch entities mentioned in passing
• Consistency checks — entities and relations must be grounded in the text, not invented
• A configurable taxonomy — entity types specific to the domain (Patent, Compound, Trial for a pharma corpus; Repo, PR, Author for an engineering corpus)

Cost anatomy

Extracting from a 1M-token corpus with Claude Haiku at $1/M input + $5/M output:

• Input: 1M tokens × $1 = $1
• Output: ~30k tokens (the extracted JSON) × $5 = $0.15
• With one round of gleaning: roughly 1.5×

So ~$1.50–$2 per 1M tokens with Haiku — cheap enough that re-indexing a corpus weekly is not unreasonable.

Pitfalls

• Hallucinated entities — the model "remembers" entities from training data. Mitigate by requiring the model to quote the source span
• Coreference — "Alice" in chunk 12 and "she" in chunk 14 are the same person. A naive extractor produces two entities; a deduplication pass fixes it (next slide)
• Granularity drift — different chunks extract the same entity at different levels of detail. Resolved at the merging stage

Sources

• microsoft/graphrag — extraction prompts & source
• Edge et al. — §3.1 Source Documents → Text Chunks → Element Instances

Slide 6 of 20Open this slide

Merging Local Extractions Into One Graph

Each chunk produces its own little entity-and-relation dump. The graph-construction stage merges all of those into a single global graph.

The merging steps

1. Normalize names

Alice ≈ Alice Chen ≈ A. Chen — resolve to one canonical entity. Two signals:

• String similarity (Levenshtein, Jaro-Winkler) for obvious typo and abbreviation matches
• Embedding similarity on the entity descriptions for semantic matches ("VP of Eng" vs "VP Engineering")
• LLM judge as the tiebreaker for ambiguous cases ("Is 'Apple Inc' in chunk 12 the same as 'Apple' in chunk 88?")

2. Merge descriptions

When two chunks describe the same entity, the system summarizes both descriptions into one via an LLM call. The merged description preserves the union of facts, drops duplication.

3. Aggregate edge weights

If chunks 1, 5, and 22 all assert "Alice manages Bob", the edge gets weight = 3 (or whatever weighting scheme — sum, max, average).

What the merged graph looks like

# Pseudocode of the resulting structure
graph = {
  "nodes": {
    "Alice": {"type": "Person", "description": "VP Engineering at Acme since 2023, oversees platform team",
              "source_chunks": [1, 7, 14, 22]},
    "Acme":  {"type": "Org",    "description": "Software company; employs ~500; uses Postgres",
              "source_chunks": [1, 7, 22, 31]},
    ...
  },
  "edges": [
    {"source": "Alice", "target": "Acme", "rel": "works_at", "weight": 3, "source_chunks": [1, 7, 14]},
    {"source": "Acme",  "target": "Postgres", "rel": "uses",  "weight": 1, "source_chunks": [22]},
    ...
  ]
}

Why the deduplication step is load-bearing

Skip it and you get graph fragmentation: 18 versions of "Alice", each with one edge — none of which can answer "how many people does Alice manage?". Most of the lift in GraphRAG comes from this stage being well-tuned.

Sources

• Edge et al. — §3.2 Element Summaries
• LightRAG — entity merging via embedding similarity

Slide 7 of 20Open this slide

Finding Clusters in the Graph

A merged knowledge graph for a real corpus has thousands of nodes. To make global queries tractable, GraphRAG runs community detection — partitioning the graph into densely-connected clusters that an LLM can summarize one at a time.

Why Leiden, not spectral or k-means

The reference GraphRAG pipeline uses the Leiden algorithm (Traag, Waltman, van Eck 2019). Three reasons:

• Hierarchical output — Leiden naturally produces a tree of communities at multiple granularities. GraphRAG uses this directly to let queries pick the "level of zoom" they need
• Resolution parameter — tunable to taste; lower resolution = bigger communities, higher = smaller
• Quality and speed — strict improvement over the older Louvain algorithm in both modularity and runtime

A typical run on a 5k-node graph: hundreds of milliseconds, producing 3 levels with 2 / 12 / 50 communities.

Summarizing each community

For every community at every level, an LLM is asked:

Here is a community of N entities and the relations among them. Produce a 1–2 paragraph summary describing what this community is about, the main entities, and notable relationships.

The output is stored alongside the graph. At query time, global search retrieves over these summaries, not over the original chunks.

Cost

Community summarization runs roughly once per (community × level). For 64 total communities across 3 levels, ~64 LLM calls totaling a few thousand input tokens each. With Haiku, single-digit dollars for the whole pass.

Tunable knobs

• Resolution — controls cluster size; defaults work for most corpora
• Levels to summarize — usually 2–3 is enough; deeper levels are diminishing returns
• Summary length — longer summaries cost more at retrieval; shorter risk losing nuance
• Min community size — too-small communities are noise; drop those below ~5 nodes

Sources

• Traag et al. — Leiden algorithm (2019)
• Edge et al. — §3.3–3.4 Graph Communities → Community Summaries

Slide 8 of 20Open this slide

Entity-Centered Retrieval

Local search answers questions about a specific entity by gathering everything the graph knows about that entity and its immediate neighbors.

What goes into the context window

For a local query, the retriever assembles roughly:

1. Description of the seed entity itself
2. Descriptions of 1-hop neighbors (and optionally 2-hop, weighted by edge strength)
3. Edge descriptions along the path between seed and neighbor
4. A handful of source chunks that mention any of the above (back-references)

All ranked by relevance, packed into the available context budget.

Code sketch

def local_search(query: str, graph, embedder, llm) -> str:
    # 1. Find seed entity by embedding similarity
    seed = embedder.nearest_node(query, top_k=1)

    # 2. Expand to neighbors
    neighbors = graph.neighbors(seed.id, max_hops=2, max_count=20)

    # 3. Gather supporting chunks
    chunk_ids = set(seed.source_chunks)
    for n in neighbors:
        chunk_ids.update(n.source_chunks)
    chunks = fetch_chunks(chunk_ids, limit=10)

    # 4. Assemble + answer
    context = format_local_context(seed, neighbors, chunks)
    return llm.generate(query=query, context=context)

When local search shines

• "Who is X?" or "What does X do?"
• "What are the dependencies of Y?"
• "What's wrong with project Z?" (recent issues, blockers, etc. that link to Z)

When it falls short

• "What are the recurring themes across all the X?" — this is global, not local
• Questions where the model doesn't know the right seed entity from the wording (e.g., "the issue we hit last sprint" — fuzzy reference)

For the second case, hybrid retrieval (next section) helps: combine an embedding-based fallback with the graph traversal.

Sources

• Edge et al. — §4.2 Local Search
• microsoft/graphrag — local search implementation

Slide 9 of 20Open this slide

Map-Reduce Over Community Summaries

Global search answers corpus-wide questions ("what are the main themes?") by asking the LLM to answer the query against each community summary, then combining the partial answers.

The two-pass pattern

Map pass

For each community summary at the chosen level, run the LLM:

You are answering the user's question using only the community summary below.
If the summary is not relevant, say so. If it is, produce a partial answer
and rate the partial answer's helpfulness on a scale of 1-100.

QUESTION: {query}
COMMUNITY SUMMARY: {summary}

This is parallelizable — N communities, N independent LLM calls.

Reduce pass

Filter to the top-K partial answers by self-rating, then ask the LLM to synthesize:

You will combine the partial answers below into one final answer.
Resolve contradictions in favor of more recent / more specific information.
Cite which community each fact came from.

QUESTION: {query}
PARTIAL ANSWERS:
- [Community 3, score 87]: ...
- [Community 12, score 76]: ...

Cost characterization

For a corpus that produced 50 communities at the chosen level, a global query costs ~50 small LLM calls in the map pass plus one larger call in the reduce pass.

With Claude Haiku and a few-hundred-token average summary, a global query costs roughly $0.05–$0.20 depending on community count. Not free; not nothing. Reserve it for queries that actually need it.

Picking the level

A query router or the user's choice picks the level. Some implementations let the model choose by retrieving summaries at multiple levels and picking the most useful.

Sources

• Edge et al. — §4.1 Global Search
• microsoft/graphrag — global search implementation

Slide 10 of 20Open this slide

The Reference Implementation

microsoft/graphrag is the canonical implementation of the Edge et al. 2024 paper — open source, Python, batteries-included.

What you get out of the box

• Indexing CLI — graphrag index --root ./project runs the whole extraction → merge → community → summarization pipeline, configurable via YAML
• Query CLI — graphrag query --method local "..." or --method global "..."
• Configurable LLMs — OpenAI, Azure OpenAI, Anthropic, local Ollama via standard adapters
• Configurable storage — Parquet on disk by default; LanceDB, Azure Cosmos DB, or your own backend
• Observability hooks — every LLM call is logged and replayable

Typical configuration

# settings.yaml
llm:
  type: openai_chat
  model: gpt-4o-mini      # or claude-sonnet-4-6 via adapter
  api_key: ${OPENAI_API_KEY}

embeddings:
  llm:
    type: openai_embedding
    model: text-embedding-3-large

entity_extraction:
  entity_types: [Person, Organization, Product, Location]
  max_gleanings: 1

community_reports:
  max_length: 2000

cluster_graph:
  max_cluster_size: 10

What it does well

• Reproducibility — outputs of every pipeline stage are versioned parquet files; you can re-run only the stage you changed
• Eval baseline — the repo ships with the eval harness from the paper, so you can compare your changes against the published numbers
• Drop-in answer quality — for global queries, "just run microsoft/graphrag with defaults" is already a strong baseline

What it does poorly

• Cost — the full pipeline on a non-trivial corpus is dollars-to-tens-of-dollars, not cents. You'll feel it
• Latency — indexing a Wikipedia-sized corpus is hours, not minutes
• Hot updates — adding a new document means re-running affected entity extraction, then re-merging; no first-class "incremental insert"

These limitations are what spawned the next wave: LightRAG, LazyGraphRAG, LinearRAG, and others.

Sources

• microsoft/graphrag
• Edge et al. — GraphRAG paper

Slide 11 of 20Open this slide

Defer the Expensive Work

LazyGraphRAG (Microsoft Research, Nov 2024) attacks the cost problem head-on: skip the up-front LLM extraction, and only do graph-style work for queries that need it.

The trade

Microsoft reports LazyGraphRAG matching GraphRAG quality at 0.1% of the indexing cost on their benchmarks — and exceeding it on some categories of global queries.

Why it works

Two insights:

1. The graph is over-extracted in eager GraphRAG. Most chunks contribute entities that no query ever asks about. The LLM work was wasted.
2. Statistical concept clusters are good enough as a routing index. You don't need a correct graph to know which slice of the corpus is relevant — embedding clustering plus a small bag-of-concepts works.

The actual graph extraction happens at query time, only over the chunks the router pulled in. Result: cheaper overall, especially on long-tail corpora.

When eager still wins

• You will query the corpus thousands of times. Amortizing the index cost is fine
• You need offline analytical workloads ("show me the dependency graph") that benefit from the full graph existing
• Latency-sensitive global queries. Lazy approaches pay an LLM-extraction cost at query time; that's hard to hide

Implementation notes

As of mid-2026 there is no official open-source LazyGraphRAG. The technique is described well enough in the Microsoft blog post to reproduce; several community implementations exist with varying maturity.

Sources

• Microsoft Research — LazyGraphRAG blog
• Edge et al. — original GraphRAG (for comparison)

Slide 12 of 20Open this slide

Dual-Level Retrieval

LightRAG (Guo et al. 2024) is a simpler, cheaper alternative to microsoft/graphrag with explicit dual-level retrieval and easier incremental updates.

The dual-level idea

Most queries have two retrieval needs at once:

• Specific — "the Postgres outage on March 14" (low-level: specific entities)
• Broad — "outages in Q1" (high-level: temporal/category clusters)

LightRAG indexes both. The LLM rewrites the query into low-level and high-level keys; the retriever does two searches and merges the results.

What's different vs microsoft/graphrag

LightRAG's main contribution isn't a new algorithm — it's a cheaper, simpler pipeline that hits similar accuracy with less infrastructure.

Incremental updates

LightRAG was designed with continuous corpora in mind. Adding a document:

1. Extract entities and relations from the new doc only
2. Find existing entities that match (string + embedding)
3. Merge; emit updated descriptions for affected nodes
4. Re-embed only the affected nodes

No global re-clustering, no community re-summarization. This is the main reason teams pick LightRAG over GraphRAG for live systems.

Reference implementation

from lightrag import LightRAG, QueryParam

rag = LightRAG(
    working_dir="./graph_storage",
    llm_model_func=claude_complete,        # any OpenAI-compatible / custom
    embedding_func=openai_embed_3_large,
)

# Indexing
await rag.ainsert(open("docs/2026-q1-report.md").read())

# Query — pick mode per call
ans = await rag.aquery("Tell me about Alice's role change",
                        param=QueryParam(mode="local"))
ans = await rag.aquery("Recurring themes from Q1 incidents",
                        param=QueryParam(mode="global"))
ans = await rag.aquery("Both",  # uses dual-level
                        param=QueryParam(mode="hybrid"))

Sources

• Guo et al. — LightRAG paper
• HKUDS/LightRAG — official implementation

Slide 13 of 20Open this slide

Memory Inspired by the Hippocampus

HippoRAG (Gutiérrez et al. 2024) borrows from neuroscience: the brain's hippocampus does pattern separation and pattern completion to retrieve memories. HippoRAG implements an analog using Personalized PageRank over an entity graph.

How PageRank fits

Personalized PageRank is the same algorithm that ranks web pages, but biased toward "seed" nodes. With seeds set to the entities in the user's query, the algorithm gives high scores to entities reachable from many of the query's seeds — a graph-aware notion of relevance.

This handles multi-hop questions naturally: if you ask about Alice and Q3 launches, entities that connect to both bubble up to the top.

What's different

In the paper's eval, HippoRAG outperforms vector RAG and matches or beats GraphRAG on multi-hop benchmarks (HotpotQA, MuSiQue, 2WikiMultiHopQA), at a fraction of the indexing cost — because there are no community summaries to compute.

Mechanism in one paragraph

Extract entities and relations from the corpus as usual. At query time, extract entities from the query; use them as Personalized PageRank seeds; run a few iterations; rank nodes by score; fetch source chunks for the top-K. Pass to the LLM.

When to reach for HippoRAG

• The query mix is heavy on multi-hop reasoning
• The corpus has lots of cross-entity relationships
• You want graph behavior without community-summarization overhead

When not to:

• The corpus is small enough that vector RAG isn't a bottleneck
• Queries are mostly point lookups
• You need explicit "themes across the corpus" — HippoRAG doesn't summarize communities

Sources

• Gutiérrez et al. — HippoRAG paper (2024)
• Personalized PageRank explainer
• OSU-NLP-Group/HippoRAG — official repo

Slide 14 of 20Open this slide

Vector + Graph + Keyword

In production, almost no one runs GraphRAG alone. The winning pattern is hybrid: a vector index, a graph index, and (often) a keyword index — combined by a small ranker.

Why hybrid wins

Each index makes a different kind of mistake:

• Vector — misses exact-match terms ("error code E_421"); fails on numerical or code-like content
• Graph — depends on entity extraction quality; fails on truly novel entities not yet in the graph
• Keyword — exact-string fragile; brittle to paraphrase

Combining them recovers most failures. The standard merger is reciprocal rank fusion: take the top-K from each retriever, score documents by Σ 1 / (k + rank_i), sort. No tuning needed.

Routing — when do you need graph at all?

A simple classifier (or a tiny LLM call) per query is cheap:

ROUTE_PROMPT = """Classify the user's query into ONE of:
- local: asks about a specific entity or document
- global: asks for themes, summaries, or aggregations across the corpus
- factual: needs an exact value, name, code, or identifier

QUERY: {query}
"""

route = classify(query)
match route:
    case "local":   return graph_local_search(query)
    case "global":  return graph_global_search(query)
    case "factual": return hybrid(vector, keyword)(query)

In practice, 70% of queries are "factual" or "local" lookups for which vector or vector+keyword is sufficient. Save graph for the global / multi-hop slice.

Production reference architecture

Most teams that ran GraphRAG in 2024–25 ended up with this shape by 2026.

Sources

• Cormack et al. — Reciprocal Rank Fusion (SIGIR 2009)
• Anthropic — Contextual Retrieval
• Microsoft GraphRAG — query routing examples

Slide 15 of 20Open this slide

What Does GraphRAG Actually Cost?

GraphRAG's LLM-heavy indexing is the headline cost. Concrete numbers for a 10 million-token corpus using Claude Haiku 4.5 (~$1 / $5 per M tokens, May 2026):

Indexing breakdown

Most of the cost is extraction. Halving the gleaning rounds, dropping low-information chunks, or using a smaller extraction model are the highest-leverage cost cuts.

Per-query cost

Local: a few thousand tokens of context, one LLM call. Cents at most.

Global: scales with number of communities at the chosen level. 5–20¢ per query is typical. This is where most of the operational cost lives in production.

How to cut cost

When the math doesn't work out

If your traffic is < 100 queries/day and your corpus is < 100k tokens, don't use GraphRAG. The indexing cost dominates per-query value. A naive vector RAG is fine.

If your traffic is > 10k queries/day with significant global-query mix, the indexing cost amortizes well — and the bottleneck shifts to query-time cost. That's when caching and LazyGraphRAG start mattering.

Sources

• Anthropic — Claude pricing
• Microsoft Research — LazyGraphRAG cost analysis

Slide 16 of 20Open this slide

Picking the Right Retrieval Strategy

Decision questions in plain English

Does your corpus have entity structure?

Yes for: customer records, code repos, knowledge bases, research libraries, project documents. No (mostly) for: blog posts, marketing copy, fiction, unstructured chat logs.

Do users ask multi-hop or global questions?

Look at real traffic. Sample 100 queries; classify. If <10% are non-local, GraphRAG's indexing cost is hard to justify.

How often does the corpus change?

Daily: LightRAG or a custom incremental pipeline. Weekly+: any GraphRAG works; re-index nightly is fine. Mostly static: even eager microsoft/graphrag is reasonable.

Cost-sensitive?

How much is "$10–$50 per index pass + few cents per query" worth to the answer-quality lift? For internal-tool quality bars, often nothing. For customer-facing search with measurable retention impact, often a lot.

Common combinations seen in production

A practical test

Before committing to GraphRAG, do this:

1. Build a vector-RAG baseline. Measure quality with LLM-judge on 50 real queries
2. Hand-annotate the 10 worst failures
3. Could those failures be answered by traversing entities or summarizing communities? If yes → GraphRAG is the right next investment. If no → fix retrieval ranking or chunking instead

Sources

• Edge et al. — GraphRAG eval methodology
• Anthropic — RAG without the cargo cult

Slide 17 of 20Open this slide

How to Measure GraphRAG

Standard RAG eval (top-K hit rate, MRR) doesn't capture what GraphRAG promises. Global queries don't have a single "correct passage" to retrieve.

The four global-query rubric dimensions

Microsoft's eval (Edge et al. §4.3) uses LLM-as-judge to score answers on four dimensions:

• Comprehensiveness — does the answer cover the breadth of the question?
• Diversity — does it surface multiple perspectives where applicable?
• Empowerment — does it help the user understand and reason further?
• Directness — does it answer the question, or wander?

The judge gets both answers (baseline vs GraphRAG) and picks a winner per dimension. Aggregate wins gives you the per-dimension lift.

Building your own eval set

You can't evaluate what you can't measure. To get a real number:

1. Sample 100–200 real queries from production logs (anonymized)
2. Classify each by question type (local fact, multi-hop, global summary, etc.)
3. Hand-label "good" answers for ~30 of them (the expensive step)
4. Run baselines and variants through the same questions
5. Score with LLM-as-judge using a fixed rubric. Use a different model from the one being evaluated, to avoid bias

For local queries, you can use exact-match or contains-match on the hand-labeled answer. For global queries, use the four-dimension rubric.

What to watch for

• Eval contamination — if your judge model trained on the corpus you're evaluating, scores are inflated
• Question shape skew — your eval should match the production query mix, not be all global queries
• Cost per eval run — running 200 queries × 4 systems × LLM-judge is real money. Cache aggressively

A minimal eval harness

@dataclass
class EvalCase:
    query: str
    type: Literal["local", "global", "multi_hop"]
    gold_answer: str | None = None

async def eval_run(systems: dict[str, RAGSystem], cases: list[EvalCase]):
    results = []
    for case in cases:
        answers = {name: await sys.answer(case.query) for name, sys in systems.items()}
        for name, ans in answers.items():
            score = await judge(case, ans)
            results.append({"system": name, "case": case.query, **score})
    return results

Sources

• Edge et al. — §4.3 Eval methodology
• Ragas — open-source RAG eval framework
• Anthropic — evaluating LLM applications

Slide 18 of 20Open this slide

Common Ways to Waste Money on GraphRAG

Anti-pattern 1: "Just turn on GraphRAG"

Running microsoft/graphrag with defaults on a corpus your team has never queried, then declaring it "improved retrieval" because anecdotes feel good.

• What's wrong: No baseline to compare against; no eval set; cost is now 100×; no one can prove (or disprove) whether it helps
• Fix: Always have a vector-RAG baseline + a measured eval set before turning on GraphRAG

Anti-pattern 2: Building a graph for a corpus that doesn't have entities

A folder of well-written technical blog posts. The pipeline runs; the graph is mostly singleton nodes; community detection produces 200 tiny communities; global queries are worse than the vector baseline.

• Fix: Look at your extracted graph. If average degree is < 2, your corpus doesn't have the right shape for GraphRAG. Stay on vector + keyword

Anti-pattern 3: Treating it as a vector-RAG replacement

GraphRAG is at its best on global and multi-hop queries; it's at best parity with vector RAG on local lookups. Replacing vector entirely makes the common case (point lookups) worse.

• Fix: Hybrid. Always.

Anti-pattern 4: Re-indexing the whole corpus on every change

A 10M-token corpus with 5 new documents/day. Re-running the full pipeline nightly. Costs scale linearly with corpus size, not change size.

• Fix: LightRAG, or roll your own incremental: re-extract only the new chunks, merge into the existing graph, re-summarize only affected communities

Anti-pattern 5: Believing the eval benchmarks transfer

The Microsoft GraphRAG paper showed +20–80% on query-focused summarization over news/podcast transcripts. Your corpus is engineering wikis and your queries are mostly "what does this code do" — those benchmarks tell you very little.

• Fix: Build your own eval set with your queries and your corpus. Always.

Anti-pattern 6: Trusting the graph too much

The LLM extracted "Alice manages Bob" from a chunk that actually said "Alice mentioned Bob in passing". The graph now has a fabricated managerial relationship. Global queries about reporting structure surface fiction.

• Fix: Require the extractor to quote source spans; sample-audit the extracted graph; weight edges by extraction confidence

Anti-pattern 7: Ignoring drift

A graph indexed in January is queried in November. Entities have been renamed, restructured, deprecated. Answers reflect history, not present.

• Fix: Time-stamp every chunk and entity; weight recent over old; re-index periodically; consider versioned graphs for audit-sensitive domains

Sources

• Edge et al. — limitations section
• Microsoft Research — LazyGraphRAG analysis

Slide 19 of 20Open this slide

Open Problems and Active Research

GraphRAG circa 2024 was a single technique. By 2026 it's an active research area with several open fronts.

Incremental and streaming GraphRAG

The biggest practical gap: efficient updates. LightRAG's per-doc incremental is a start; LazyGraphRAG sidesteps the problem differently; nobody has cracked "continuous streaming corpora with sub-second update latency to the graph". Active work in 2026 papers.

Cross-corpus graphs

Most graphs are per-corpus, per-tenant. A common ask in enterprise: "graph that spans my internal wiki, my customer-support tickets, and my code." Tooling for safe cross-corpus graphs (auth, deduplication, governance) is immature.

Causal and temporal edges

Today's GraphRAG edges are mostly co-occurrence: "Alice and Bob are mentioned together." Causal edges ("the postponement of Project X caused the reorg of Team Y") and temporal edges ("Alice managed Bob from Jan to Aug; now Alice reports to Carol") are not first-class. Several papers in 2025 propose typed relation extraction with temporal grounding.

Multi-modal graphs

The text-extraction pipeline ignores images, diagrams, code, tables. A diagram in a slide deck is a graph; extracting and merging it with the text graph is open. The 2026 frontier models that handle high-resolution vision (Claude Opus 4.7) make this newly tractable.

Learned routing

The "local vs global" router is usually hand-tuned. Several papers propose learning the router from query/answer pairs, with measurable gains on heterogeneous workloads.

Compact graph representations

A 5M-node graph doesn't fit in a context window. Current approaches summarize communities; future approaches might compress the graph itself into a learned representation that the model can attend to directly. Early work on this is promising but very early.

What you can take to industry now

• microsoft/graphrag for static, single-corpus, batch-indexed
• LightRAG for incremental and cost-sensitive
• HippoRAG for multi-hop-heavy workloads
• Hybrid with vector + keyword in all cases
• A measured eval harness in all cases

Sources

• Edge et al. — GraphRAG paper
• Microsoft Research — LazyGraphRAG
• Guo et al. — LightRAG
• Gutiérrez et al. — HippoRAG

Slide 20 of 20Open this slide

What to Remember

Five claims

1. GraphRAG indexes relationships, not just chunks. The LLM does extra work at index time so retrieval can traverse a graph at query time
2. Local and global search are different code paths. Local = entity-centered drill-down. Global = map-reduce over community summaries
3. The biggest cost is indexing, not querying. Most production trouble is about making that cost amortize
4. Hybrid wins. Vector + graph + keyword + reranker is the actual production architecture. Pure GraphRAG is for special-purpose workloads
5. The benchmarks don't transfer. Build your own eval set with your queries and corpus before committing — and especially before claiming GraphRAG "works" for your case

What you should be able to do after this lecture

• Sketch the indexing pipeline (chunks → extract → merge → cluster → summarize)
• Decide whether a query is local or global
• Read a community summary and explain how it would be used in global search
• Pick between microsoft/graphrag, LightRAG, LazyGraphRAG, HippoRAG given constraints
• Identify cost and quality trade-offs of each
• Write a 50-line PageRank-on-entity-graph baseline if asked

What's next

• Week 7 — Evaluation & LLM-as-a-Judge. The eval harness you need to actually measure whether GraphRAG (or any retrieval change) helps
• Week 11 — HITL & Memory. Long-term memory is graph-shaped; many of the same patterns apply

Sources

• Edge et al. — GraphRAG paper
• Microsoft Research — LazyGraphRAG
• Guo et al. — LightRAG
• Gutiérrez et al. — HippoRAG
• microsoft/graphrag
• HKUDS/LightRAG