Knowledge graph or vector search for RAG?

Separate storage, retrieval and the final answer

RAG combines a generation model with retrieved information. The original RAG paper explores this combination using an external document index. A deployed system still needs a clear rule for which material is eligible to support an answer.

A graph describes entities and relationships. The W3C RDF primer introduces statements expressed as subject, predicate and object. That representation is useful when the relationship itself matters, rather than merely the similarity of two pieces of text.

Start with actual questions

A request to find documents about a topic differs from a request to identify which service depends on a specific component. The first may benefit from semantic retrieval; the second calls for reliable entity identity and explicit relationships.

Write representative questions before choosing a database. Mark the source that supports each expected answer and note when an answer requires combining several records. This becomes the basis for testing retrieval separately from generation.

• Collect questions in the language users actually use.
• Identify the source of each expected answer.
• Include aliases, outdated material and missing information.
• Separate a relevant passage from sufficient evidence.

Source quality remains a separate problem

Neither embeddings nor graph edges establish that an input document is current. Define who owns source updates, what the date means and how withdrawn documents stop appearing in answers. Preserve a route from an answer back to its supporting material.

For private collections, access filtering belongs in the retrieval boundary. A useful answer must be both supported and permitted for the requesting user. Testing only an administrator account leaves this part of the product unexamined.

Compare approaches on the same evidence

Evaluate a simple search baseline, semantic retrieval and a graph-assisted alternative on the same question set. Inspect missing sources, incorrect relationships and cases where the answer should explicitly remain unknown.

Record retrieval coverage, the correctness of cited support, accepted-answer rate and response time. These measurements answer different questions. Finding more documents can increase workload for the model without improving the final result.

Account for maintenance, not just the first demonstration

An explicit graph introduces work around entity identity, relationship definitions and updates. A vector index also requires source ingestion, refresh and evaluation. Compare who maintains each system and how errors are corrected.

A combined approach is reasonable when its benefit appears in the evaluation. It is unnecessary when simpler retrieval already satisfies the task. Keep the choice reviewable by retaining the question set and documenting the cases where each approach failed.