Ontology plus vector search for GraphRAG means using structured domain knowledge and semantic retrieval together to build better context for a language model.
The ontology defines concepts and relationships. Vector search finds semantically relevant text or entities. GraphRAG uses the graph layer to retrieve connected evidence that ordinary chunk search may miss.
Short Answer
Use an ontology to define the domain model, annotate documents and entities with ontology concepts, store embeddings for chunks or entities, then use vector search to find relevant entry points into a knowledge graph.
From those entry points, traverse relationships, collect source chunks, include entity and relationship summaries, and assemble grounded context for RAG.
Why GraphRAG Needs More Than Vectors
Vector search is useful because it retrieves by meaning.
But a vector index usually treats chunks as independent items. It does not automatically know that one entity owns another, that one policy depends on another, or that two events are connected through a shared actor.
GraphRAG adds those relationships to retrieval.
What the Ontology Does
The ontology provides the conceptual contract for the graph.
It defines what kinds of entities exist, what relationships are valid, what properties matter, and how concepts are grouped.
Without this layer, extracted graph data can become inconsistent and hard to query.
What Vector Search Does
Vector search finds semantically relevant chunks, documents, or entities.
It handles paraphrases, fuzzy user questions, and natural language that does not exactly match the stored text.
In GraphRAG, vector search is often used to find the best starting points for graph retrieval.
What the Graph Does
The graph stores relationships between entities, concepts, source chunks, and summaries.
It can answer questions that depend on connections rather than only similar text.
For example, it can retrieve entities connected by ownership, dependency, membership, cause, citation, contract, policy, location, or sequence.
The Three-Part Model
A practical GraphRAG retrieval system has three parts:
- ontology for domain structure
- vector search for semantic discovery
- graph traversal for connected context
The value comes from coordinating these parts rather than treating them as separate search systems.
Indexing Pipeline
The indexing pipeline usually starts with source documents.
Documents are chunked, embedded, and stored for semantic search.
Entities and relationships are extracted from the same documents, normalized against the ontology, and stored in a graph with links back to source chunks.
Entity Extraction
Entity extraction identifies people, organizations, products, concepts, places, issues, policies, risks, or other domain objects.
The ontology should define which entity types are allowed.
This keeps extraction focused on what the retrieval system actually needs.
Relationship Extraction
Relationship extraction identifies how entities are connected.
Examples include “works for,” “depends on,” “mentions,” “is part of,” “causes,” “treats,” “supersedes,” “cites,” or “is governed by.”
Relationships should be typed, sourced, and auditable.
Concept Normalization
Extracted labels need normalization.
The same entity or concept may appear under aliases, abbreviations, spelling variants, or translated names.
Ontology IDs help merge these forms into stable concepts.
Chunk Links
Graph nodes should link back to the chunks that support them.
This is important for RAG because the language model needs source evidence, not only graph labels.
Chunk links also make it easier to audit wrong answers and bad extractions.
Embedding Chunks
Chunks are embedded so the system can find relevant source text.
This supports ordinary semantic retrieval and provides evidence for generated answers.
Chunk embeddings are useful when the query is mostly about text content.
Embedding Entities
Entities can also be embedded.
An entity embedding may represent the entity name, aliases, description, properties, or summary.
This lets vector search find relevant graph nodes directly from a natural-language query.
Embedding Summaries
GraphRAG systems often create summaries for entities, relationships, or communities.
Embedding those summaries can make graph retrieval more effective because they compress distributed information into searchable text.
Summary embeddings can act as bridges between natural language queries and graph structure.
Query Pipeline
A query pipeline typically follows these steps:
- interpret the query
- identify concepts and entities
- run vector or hybrid search
- select graph entry points
- traverse relationships
- collect source evidence
- rerank and assemble context
- generate an answer with citations
Entity Entry Points
Entity entry points are graph nodes selected as starting points for retrieval.
Vector search can find these nodes even when the query does not use the exact entity name.
Once selected, the graph can expand from those nodes to connected evidence.
Local Graph Search
Local graph search retrieves a neighborhood around relevant entities.
It may include connected entities, relationships, source chunks, entity summaries, and community summaries.
This is useful when the question is about a specific entity or concept.
Global Graph Search
Global graph search is useful when the question asks about broad themes across the corpus.
Instead of focusing only on one entity neighborhood, it may use community summaries or higher-level graph structures.
This helps with questions that require synthesis across many documents.
Ontology-Guided Traversal
The ontology can decide which graph edges are useful for a query.
A question about risk may prioritize relationships such as “depends on,” “violates,” “requires,” or “impacts.”
A question about authorship may prioritize “wrote,” “approved,” “signed,” or “cited.”
Traversal Depth
Traversal depth controls how far the system walks from the starting entities.
One hop may be enough for direct facts. Two or three hops may be needed for dependency chains or indirect relationships.
Depth should be limited because graph expansion can quickly add noise.
Hybrid Search
Hybrid search combines vector similarity with keyword matching.
It is useful in GraphRAG because entity names, codes, acronyms, contract IDs, citations, and product names often require exact matching.
Hybrid retrieval helps catch both semantic meaning and precise identifiers.
Metadata Filters
Metadata filters keep retrieval inside the correct scope.
Filters may include tenant, access role, product, source type, region, language, document status, date, confidentiality, or ontology concept.
These filters should be enforced before context is shown to the language model.
Reranking
Reranking combines retrieval signals into a final order.
A strong reranker can consider vector score, keyword score, ontology match, graph distance, relationship type, source authority, recency, and evidence quality.
This helps select the best context when vector and graph signals disagree.
Context Assembly
The final RAG context should be concise and sourced.
It may include top chunks, entity summaries, relationship summaries, graph paths, community summaries, and key metadata.
The goal is to give the model enough connected evidence to answer without overloading the prompt.
Provenance
Every graph fact should point back to evidence.
Provenance can include source document IDs, chunk IDs, extraction timestamps, confidence scores, and ontology versions.
This makes GraphRAG answers easier to verify and debug.
When This Architecture Helps
Ontology plus vector search helps GraphRAG when questions depend on relationships.
Examples include compliance obligations, research networks, contract relationships, product dependencies, medical pathways, legal citations, customer journeys, supply chains, and organizational knowledge.
It is less necessary when simple semantic chunk retrieval already answers the query.
When Metadata Is Enough
A full graph is not always required.
If the system only needs categories, filters, synonyms, and facets, metadata fields may be enough.
Use a graph when relationship traversal changes the answer.
Schema Design
Good schema design separates documents, chunks, entities, concepts, relationships, and summaries.
Each object should have stable IDs, source links, timestamps, and version metadata.
This avoids mixing raw text, extracted facts, and generated summaries into one ambiguous object.
Ontology Versioning
The ontology will evolve.
Concepts may be renamed, split, merged, or deprecated.
Versioning helps keep graph nodes, metadata annotations, and evaluation results consistent over time.
Quality Checks
Quality checks should cover both retrieval and graph construction.
Evaluate entity extraction, relationship extraction, concept normalization, vector recall, graph traversal relevance, context faithfulness, and answer quality.
GraphRAG quality depends on the whole pipeline, not only the vector index.
Common Failure Modes
- extracting too many irrelevant entity types
- missing aliases and duplicate entities
- storing relationships without source evidence
- using graph traversal without depth limits
- ranking only by vector similarity
- overusing community summaries without citations
- failing to enforce permissions across graph edges
- letting ontology versions drift from indexed data
Security and Access Control
Graph retrieval can reveal information indirectly.
If a user cannot access a document, the system must also hide graph facts and summaries derived from that document.
Access rules need to apply to chunks, entities, relationships, summaries, and generated context.
Operational Monitoring
Monitor retrieval quality after deployment.
Track empty results, low-confidence answers, graph expansion size, context size, citation quality, latency, and user feedback.
GraphRAG systems can degrade when the corpus, ontology, or extraction model changes.
Practical Starting Point
A practical starting point is to annotate chunks with entity and concept IDs, embed both chunks and entity summaries, and store relationships in a small graph.
Use vector search to find candidate chunks or entities, then traverse only the most relevant relationships.
Expand gradually when evaluation shows that graph context improves answers.
Summary
Ontology plus vector search for GraphRAG combines three strengths: domain structure, semantic retrieval, and relationship-aware context.
The ontology defines the meaning of entities and relationships. Vector search finds relevant text or graph entry points. Graph traversal gathers connected evidence for richer RAG answers.
The architecture works best when it preserves provenance, controls graph expansion, enforces access rules, and evaluates retrieval quality with real questions.