GraphRAG API: Optimize your LLM with Knowledge Graphs

Posted Monday, May 12, 2025.

GraphRAG offers a highly efficient way to improve the performance of your LLMs by using knowledge graphs to generate context-aware responses. LLM's own knowledge may be too general or limited and traditional RAG methods may not work well for general queries and won't capture complex relations. GraphRAG uses a knowledge graph to improve the original user queries and capture complex relations in data.

There are multiple different types of GraphRAG systems available and they all function in different ways as there's not a single standard for how they should work. The advantage of the InfraNodus GraphRAG API is that it provides a visual interface that can be used for contextual overview and targeted queries. It also has an easy-to-use API that can be used for prompt augmentation, get relational context for your RAG queries, generate content gaps, and AI responses that bridge those gaps.

GraphRAG vs Traditional RAG: Advantages

Traditional RAG (retrieval-augmented generation) is used to provide additional context to a general-purpose LLM. The user asks a question, it is converted into a vector representation and is then matched to the most similar documents in the knowledge base (that are also represented by multidimensional vectors). The retrieved document chunks are then added to the model's context and are used to provide an answer. There are multiple problems with this approach.

First, when the user query is too general (e.g. "What is it about?"), standard vector similarity search will not understand user's intent to get an overview of the document. Instead, it will find the chunks that are most similar to the terms "what", "it" and "about" (which can be anything) and provide the most irrelevant context to the model.

Another problem is that traditional RAG does not capture relations in text, so when you make complex queries like "What are the approaches I should be using with this framework?", RAG will look for "framework" and "approaches", not what they are connected to.

Using a knowledge graph helps solve these problems. It encodes the complex relationships and can be used to retrieve relations between the entities. As there is a model of the whole text structure, it can also be used to retrieve an overview, identify topical clusters, and add this as additional context to the model. These insights can be used both to augment users' prompts or to provide additional context to the model to improve its responses.

For example, when you ask a model about the main topics in a document, it will use its underlying representation of the whole dataset and retrieve the main topical clusters, which can be used to provide better results. When you ask about a specific relations, it will retrieve a relevant part of the graph and traverse it several hops away to retrieve all related statements that may not be semantically similar. It will then provide those text chunks to the context and increase model's awareness of the context in a way that is specific to the particular dataset you're analyzing.

Typical Graph RAG Applications

There are different types of Graph RAG applications that can be used for different purposes. Often, they are combined for better results.

1. User Prompt Augmentation

One of the most typical use cases for GraphRAG is user prompt augmentation. In this scenario, GraphRAG is added to an LLM pipeline in order to add contextual information to the user's prompt, which can then be used with the standard RAG pipeline to generate better answers.

For example, if the user asks "What is it about?", before sending it to a knowledge retrieval function, we can retrieve the general information about the text — the main topical clusters, concepts, and relations — and ask an LLM to improve user's propmt with that contextual information. So it becomes "What are the most important ideas in this document related to text analysis, API, and data visualization?". When this augmented prompt is then sent to a standard RAG pipeline, vector search will retrieve much more relevant results as we're providing more context in our query. This context will then be used to generate a better response for the user.

Further reading: prompt augmentation for LLM RAG with Dify and InfraNodus.

2. Adding Relations to Context

Another common use case for GraphRAG is to provide relational context to LLMs. In this scenario, GraphRAG will retrieve the most relevant parts of the knowledge graph related to the user's query. These parts may be encoded as ontological relations (e.g. "Text Analysis" [used in] "Data Science"), as it is done in the Neo4J, LlamaIndex, and InfraNodus GraphRAG systems. They may also contain the actual statements, which will be added to the model's context — as it is done in InfraNodus. This additional relational data will help the model generate more focused results with a particular emphasis on ontological relations present in a particular dataset.

Further reading: Portable GraphRAG.

3. Topical Overview and Summarization

GraphRAG can also be used to store a topical overview of a document or a dataset. In this scenario, GraphRAG identifies the main clusters of entities present in text and then organizes them into topics, which can then be summarized and used for providing additional context to the model. This approach is used in Microsoft and InfraNodus GraphRAG.

Further reading: using GraphRAG to generate topical clusters and summaries.

4. Knowledge Base Optimization

Knowledge base optimization is another interesting use case for GraphRAG. Knowledge base is used to retrieve contextual data for LLM responses. Usually it's a collection of documents or support articles to be used for response generation. GraphRAG can be used to evaluate the quality of this knowledge base and to see whether something is missing. This can be done using the InfraNodus GraphRAG, which will visualize the main topical clusters in a graph and show the relations between them. This representation can be used to assess the completedness of the knowledge base and identify the content gaps that need to be filled in order to improve the quality of retrieval.

Further reading: knowledge base optimization.

5. Content Gap Detection

Knowledge graphs can be used to detect content gaps in a dataset. These can be identified based on the structural holes in the knowledge graph: the topical clusters that could be better connected but are not. Bridging these gaps can help generate interesting but relevant insights because they connect the ideas that already exist in a text in a completely new way. InfraNodus knowledge graphs can be used for this purpose and there is a dedicated endpoint in InfraNodus API that provides this insight based on the text's knowledge graph structure. Content gap detection can be integrated into your automation workflows as additional tools that can help remove model's bias and point its attention to what's missing in a particular text.

Further reading: content gap detection for agentic workflows with Crew AI.

GraphRAG Comparison: Neo4J vs Microsoft vs InfraNodus

There are several GraphRAG frameworks available in the market today and they work slightly differently. They also differ in terms of complexity and ease of use. Choosing the right framework depends on your objectives. Below we overview the most popular ones and compare them to one another.

Neo4J GraphRAG

Neo4J is a popular graph database that that be used for knowledge graph storage and retrieval. It is relatively easy to set up on the cloud and it has multiple tutorials.

The problems start when you want to generate a graph. You can use their LLM graph builder, but it takes a long time to process the data and might use a lot of your tokens as it has to process the whole document, feed it to an LLM of your choice (that you will provide your own API key for), and then generate Cypher queries (Neo4J database language) to create the graph. This is a very long and tedious process. As a result, you will have a graph of entities and ontological relations and can then use the Neo4J's GraphRAG library (directly in Python or in your favorite framework, such as LangChain or LlamaIndex) to extract those relations and augment your prompts with them. However, as you can see, you still need a bunch of external frameworks to use it, it's hard to update the knowledge base with new data, and it can be costly, especially if you don't host everything yourself.

Pros:
• One of the market leaders
• Has a basic graphical interface
• Has cloud hosting options to try it out
• Can build large graphs and encode complex ontologies
• Integrates with popular LLM frameworks

Cons:
• Difficult to generate graphs
• Graphs can be expensive to encode
• Difficult to update
• Lack of analysis / editing features in the graph interface
• Requires extensive coding knowledge to use in an API

Microsoft GraphRAG

Microsoft GraphRAG is one of the most popular solutions, but it is not as flexible as the other options. It's based on generating a knowledge graph for your data, identifying community clusters, and then generating summaries for those clusters to provide better context to your responses.

However, Microsoft GraphRAG is very difficult to set up. You could theoretically host it yourself, but it's too buggy, so you need to set up a new Azure instance and deal with permissions and API keys. Once you set it all up, it will take you hours to launch the Microsoft GraphRAG instance due to multiple Python incompatibilities and dependencies. It may be a good solution for enterprise users who have a team to dedicate to setting it up along with a paid Azure support plan, but it's not a good solution for mere mortals.

Pros:
• Conceptually interesting
• Can be hosted on Azure
• Suitable for enterprise users

Cons:
• Requires complex set up
• Needs coding knowledge
• No graph interface
• Lack of observability
• Can be expensive to run on Azure

LangChain Graph RAG Retrieval

LangChain is a popular framework for building LLM applications. It provides a Graph RAG retrieval component that can be used to emulate graph retrieval based on metadata properties of your text chunks.

Technically, LangChain Graph RAG is not really a knowledge graph because it doesn't have a holistic view of the content. Rather, it is an ad-hoc solution where you can have graph-like retrieval by specifying certain metadata properties as the graph edges (e.g. link the text chunks, which have the same "location" property) and improve RAG retrieval using those edges to improve the precision. However, as it does not have a coded ontology or general representation of the content, it can only be used to add some relational data to prompts based on the properties, but it cannot be used for knowledge base optimization, prompt augmentation, or content gap detection.

Pros:
• Relatively simple to set up
• Uses existing metadata in your documents
• Integrated in a popular framework
• Open-source

Cons:
• Is not a proper knowledge graph
• Does not have a holistic view of the content
• No graph interface
• Lack of relation descriptions

InfraNodus GraphRAG

InfraNodus GraphRAG is a simple and easy to use solution that allows you to generate a knowledge graph for your data, identify community clusters, and then generate summaries for those clusters to provide better context to your responses. It can also retrieve relational data based on the users' query and identify structural gaps in your knowledge, which can be used to generate insights and recommendations.

InfraNodus provides an easy-to-use knowledge graph API that can be integrated into any LLM pipeline, framework, or applications (such as Dify for building chatbots, for instance). It also has an interactive graph that can be used to explore and clean the knowledge base and to emulate user queries in order to understand which parts of the graph are used to provide responses.

Pros:
• Easy to use
• No need to code
• Interactive graph
• Easy to update

Cons:
• Requires subscription
• May have limitations on the total knowledge base size
• Does not support local Ollama GraphRAG (yet)

Try It Yourself

You can try the InfraNodus GraphRAG API after you sign up for an account and get an API key: