In this series of articles, we will explain Large Language Models, Knowledge Graphs and their combinations to examine the popular patterns of combining them and finally discuss to what extent this patterns will persist or perish in the future.

These articles are co-written by Anis Aknouche and Ole Olesen-Bagneux, the first one dives into LLMs, the second dives into KGs, and the third dives into their synergy. All three articles contain tangible ways that are useful for enterprise data management and science.

Figure 1. The three LLM + KG patterns.

This article is part of a series of 3 articles to explain LLM + KG patterns. In this last part (3/3), we will talk about how Knowledge Graphs (KGs) and Large Language Models (LLMs) can mutually enhance each other in the same framework.

You can find the first article (1/3), where we talk about Large Language Models (LLMs) and how they can be enhanced using Knowledge Graphs (KGs) at KG-enhanced LLM (part 1/3), and the second article (2/3) where we talk about how Knowledge Graphs can be augmented using LLMs at LLM-augmented KG (part 2/3).

1. Synergized LLM + KG

Figure 2. Synergized LLM and KG Architecture.

In the previous two articles, we explored two complementary but one-directional integration patterns: KG-Enhanced LLMs, where structured knowledge is injected into language models, and LLM-Augmented KGs, where language models help build, complete, or query knowledge graphs. Both approaches are powerful in their own right, but they treat the relationship between LLMs and KGs as a one-way integration.

This article takes a different stance. Rather than asking “how can one technology serve the other?”, we ask: “how can they serve each other simultaneously?” This is the core idea behind Synergized LLM + KG, a fully bi-directional fusion where each modality continuously informs and improves the other.

The synergy of LLMs and KGs, as defined in Pan et al. [1], frames LLMs and KGs as a unified, mutually reinforcing system. LLMs provide KGs with language understanding and generalization capabilities, while KGs provide LLMs with explicit, structured, and interpretable knowledge. Neither technology is subordinate to the other, they act as dual engines driving a single, more capable system.

2. Synergized LLM + KG Approaches

The synergized paradigm materializes through two main technical directions: 1) Unified Knowledge Representation and 2) Joint Reasoning. Let’s walk through each.

2.1. Unified Knowledge Representation

The first direction focuses on learning shared representations that simultaneously encode both textual information and graph-structured knowledge. Rather than keeping language and graph modalities separate and only bridging them at inference time, these approaches train models that natively use both signals.

ERNIE [2] is one of the earliest and most influential examples. It introduces a textual-knowledge dual encoder architecture composed of two components: a T-Encoder, which processes the input sentence using transformer-based language modeling, and a K-Encoder, which separately processes a relevant subgraph from a knowledge graph. The two encoders are then fused, allowing entity mentions in the text to be grounded in their corresponding KG representations. The result is a model that understands both the linguistic context of words and the structured relationships between the concepts they refer to.

DRAGON [3] pushes this idea further with a self-supervised pre-training approach that jointly learns from text and KG data without requiring any labeled examples. The model takes as input a text segment alongside a relevant KG subgraph, and bidirectionally fuses information from both modalities throughout the encoding process. To optimize its parameters, DRAGON is trained on two complementary tasks: masked language modeling, which teaches the model to understand text, and KG link prediction, which teaches it to reason about relational structure. This dual pre-training objective forces the model to develop representations that are simultaneously grounded in language and in structured knowledge.

For enterprises, this kind of approach has real value, imagine a model pre-trained on your internal documentation and your company’s ontology at the same time. Queries that require both understanding natural language intent and navigating structured data relationships become tractable in a single forward pass.

2.2. Joint Reasoning

The second direction focuses not on representation learning but on reasoning: how can LLMs and KGs collaborate to answer complex, multi-hop questions that neither could handle well alone?

Two distinct strategies have emerged here. The first tightly integrates the two modalities within a shared architecture. The second keeps them separate and uses the LLM as an intelligent agent that queries the KG on demand.

2.2.1. LLM-KG Fusion Reasoning

In this strategy, the LLM and the KG are wired together so that they can interact at a fine-grained level during inference. The intuition is that when a language model processes a question, it should be able to “look up” relevant KG entities and relations in real time, and those entities should in turn influence how the model attends to the text.

JointLK [4] formalizes this intuition through a framework that enables fine-grained, token-level interaction between the language model and the knowledge graph. Specifically, it introduces a bi-directional attention mechanism with two components: LM-to-KG attention, which allows tokens in the text to attend to relevant KG entities, and KG-to-LM attention, which allows KG entities to attend back to the text tokens that are most relevant to them. This two-way flow of information allows the model to continuously align its linguistic understanding with the structured knowledge available in the graph, leading to more precise and grounded answers.

GreaseLM [5] takes a similar philosophy but implements it more deeply. Rather than adding an interaction layer after the LM, GreaseLM integrates rich, layer-wise interactions between text tokens and KG entities at every layer of the transformer. As the model processes input at each depth, it exchanges information with the graph, making the reasoning process progressively more informed by structured knowledge. This design is particularly effective for commonsense question answering, where answers often depend on implicit knowledge that must be retrieved from a graph rather than inferred from the text alone.

Both approaches share a key insight: the most powerful fusion is not a post-hoc bridge between two separate systems, but a deep architectural integration where language and knowledge co-evolve during inference.

2.2.2. LLMs as Agents Reasoning

A second and increasingly popular strategy takes a more modular approach. Rather than fusing LLMs and KGs at the architecture level, it uses the LLM as an intelligent agent that decides when and how to query the KG, interprets the results, and incorporates them into its reasoning chain. This is closer in spirit to how a human expert uses a reference database: not by memorizing it, but by knowing how to search it.

KSL [6] (Knowledge Selective Learning) teaches LLMs to actively search a knowledge graph for relevant facts before formulating an answer. Instead of relying solely on parametric knowledge baked into the model’s weights, KSL trains the LLM to issue targeted queries to the KG, retrieve the most relevant triples, and integrate that information into its response. This reduces hallucination and improves factual reliability, especially in domains where the underlying knowledge changes over time.

StructGPT [7] takes a more programmatic approach by designing a set of structured API interfaces that allow an LLM to directly access and navigate relational data. When faced with a question that requires traversing a KG, the LLM can call these APIs to retrieve neighbors, filter by relation type, or follow multi-hop paths, all within a single reasoning session. The strength of this approach is its generality, the same LLM can be used across different KGs simply by providing the appropriate API specification.

Think-on-Graph [8] goes even further by framing KG-grounded reasoning as a beam search problem. Given a question, an LLM agent iteratively explores the knowledge graph by selecting the most promising paths at each step, using the LLM’s language understanding to score and prune candidates. This process continues over multiple hops until a confident answer is found. Think-on-Graph is particularly compelling because it is fully plug-and-play: any LLM can be used as the reasoning engine, and any KG can serve as the knowledge backbone, without requiring any model fine-tuning.

Together, these agentic approaches represent a pragmatic and scalable path toward synergized LLM + KG systems, one that is already finding its way into enterprise search, data exploration, and automated knowledge base maintenance.

The synergy of LLMs and KGs is still an active and evolving research frontier. The approaches described above illustrate a rich design space: from tightly coupled architectures that fuse representations at training time, to loosely coupled agent systems that collaborate at inference time. Each point in this space comes with its own trade-offs in terms of accuracy, interpretability, scalability, and ease of deployment.

What is clear is that neither technology alone is sufficient for the most demanding enterprise applications. LLMs are powerful but opaque and prone to hallucination. KGs are precise but brittle and expensive to maintain. Their combination, when done thoughtfully, can address the weaknesses of each.

3. Challenges of Synergized LLM + KG

Despite the promise of synergized LLM + KG systems, several important challenges remain that both researchers and practitioners should be aware of.

  1. Scalability is the first concern. Deep fusion approaches like GreaseLM and JointLK are computationally expensive. The tight coupling between LLM layers and KG entities requires running graph neural network operations at every transformer layer, which significantly increases memory usage and inference time. Scaling these approaches to large enterprise KGs with millions of entities is non-trivial.

  2. Up to date knowledge is a second challenge. KGs are snapshots of knowledge that require ongoing curation to stay up to date. In real-world deployments, there is always a lag between the world changing and the KG reflecting that change. LLMs, trained on static corpora, face the same problem. A synergized system inherits the staleness of both, and no existing approach fully solves the problem of continuous, low-friction knowledge update.

  3. Alignment and grounding remain difficult. Even in well-designed fusion architectures, it is hard to guarantee that an entity mention in text correctly maps to the right node in the graph, especially when entity names are ambiguous, domain-specific, or expressed in ways the model was not trained on. Poor alignment leads to noise being injected into the reasoning process rather than signal.

  4. Interpretability is also a concern. While KGs are inherently interpretable (a fact is a triple: subject–relation–object), the fusion of KG representations with LLM activations produces intermediate states that are no longer human-readable.

  5. Evaluation is hard. Most benchmarks used to evaluate synergized LLM + KG systems focus on question answering tasks that may not reflect the complexity and messiness of real enterprise data. Progress on these benchmarks does not always translate directly into production-ready improvements.

Conclusion

KGs and LLMs are not competing technologies, they are complementary ones. LLMs bring language understanding, generalization, and the ability to reason over unstructured text. KGs bring structure, precision, and interpretable relational knowledge. Alone, each has significant blind spots. Together, they can cover each other’s weaknesses in ways that neither can achieve independently.

The synergized LLM + KG pattern explored in this article represents the most ambitious expression of this complementarity. Rather than letting one technology serve the other in a one-directional handoff, synergized systems allow both to co-evolve: the LLM grounds its reasoning in the KG’s structured facts, while the KG benefits from the LLM’s ability to navigate language, ambiguity, and context.

To fully realize this potential, the field will need to incorporate additional advances, including multimodal learning to handle images and tables alongside text and graphs, graph neural networks to improve the expressiveness of KG encoders, and continual learning to keep both LLMs and KGs current without costly full retraining.

The practical upside is substantial. Synergized systems are already being applied to enterprise search, where both semantic relevance and factual accuracy matter, to recommender systems, where structured user and item attributes must be combined with behavioral signals; and to drug discovery, where navigating vast biomedical graphs and interpreting scientific literature are inseparable tasks.

The dual-engine metaphor is apt: with knowledge-driven graph search running in parallel with data-driven language inference, and with each engine validating and informing the other, synergized LLM + KG systems can tackle problems that single-modality approaches simply cannot. We expect this integration pattern to attract growing attention and growing adoption in the years ahead.

Key Takeways

Synergized LLM + KG important advantages:

  • Continuous Bi-directional Knowledge Enrichment

  • Grounded Reasoning with Natural Language Fluency

  • Reduced Hallucination with Verifiable, Traceable Outputs

  • Unified Representation of Structured and Unstructured Knowledge

  • Adaptive Knowledge Systems

Thanks for reading The Data Intelligence Platform! Subscribe for free to receive new posts and support our work.

References

[1] PAN, Shirui, LUO, Linhao, WANG, Yufei, et al. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 2024, vol. 36, no 7, p. 3580-3599.

[2] ZHANG, Zhengyan, HAN, Xu, LIU, Zhiyuan, et al. ERNIE: Enhanced language representation with informative entities. In : Proceedings of the 57th annual meeting of the association for computational linguistics. 2019. p. 1441-1451.

[3] YASUNAGA, Michihiro, BOSSELUT, Antoine, REN, Hongyu, et al. Deep bidirectional language-knowledge graph pretraining. Advances in Neural Information Processing Systems, 2022, vol. 35, p. 37309-37323.

[4] SUN, Yueqing, SHI, Qi, QI, Le, et al. JointLK: Joint reasoning with language models and knowledge graphs for commonsense question answering. In : Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lin.

[5] ZHANG, Xikun, BOSSELUT, Antoine, YASUNAGA, Michihiro, et al. Greaselm: Graph reasoning enhanced language models for question answering. arXiv preprint arXiv:2201.08860, 2022.

[6] FENG, Chao, ZHANG, Xinyu, et FEI, Zichu. Knowledge solver: Teaching llms to search for domain knowledge from knowledge graphs. arXiv preprint arXiv:2309.03118, 2023.

[7] JIANG, Jinhao, ZHOU, Kun, DONG, Zican, et al. Structgpt: A general framework for large language model to reason over structured data. In : Proceedings of the 2023 conference on empirical methods in natural language processing. 2023. p. 9237-9251.

[8] SUN, Jiashuo, XU, Chengjin, TANG, Lumingyuan, et al. Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. arXiv preprint arXiv:2307.07697, 2023.