Newest papers

An Empirical Study of Recent Large Language Models for Knowledge Graph to Text Generation on Standard Benchmarks

Priyanka, Renaud Delbru, Souhail Bakkali, Mickaël Coustaty, Antoine Doucet · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31529 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

知识图表到文本(KG-to-text)生成将一组知识图表三倍化为流畅的自然语言。虽然精密调节的编码器和解码器模型长期以来一直是标准方法,但最近的LLM正在变得越来越有竞争力。我们将最近的LLM与精密调节的编码器和解码器模型进行比较,使用基于参考的指标和基于图表的基于参考的LLM评估进行验证,对人类注释进行了验证。我们观察到,最近的LLM在几个设置中显著狭窄,在基于参考的指标上与精密调节的模型之间的差距正在缩小,同时在基于图表的指标上表现更好。基于LLM的评估得分显示与基于参考的指标有限相关性

Machine translation (Meta M2M100); the English original follows.

Abstract

Knowledge graph-to-text (KG-to-text) generation verbalizes a set of knowledge graph triples into fluent natural language. While fine-tuned encoder-decoder models have long been the standard approach, recent LLMs are becoming increasingly competitive. We benchmark recent LLMs against fine-tuned encoder-decoder models, using reference-based metrics and graph-grounded reference-free LLM-based evaluation validated against human annotations. We observed that recent LLMs substantially narrow, and in several settings close, the gap with fine-tuned models on reference-based metrics, while performing better on graph-grounded metrics. LLM-based evaluation scores show limited correlation with reference-based metrics, suggesting that the two approaches capture different aspects of generation quality. Graph-grounded evaluation reveals that prompted models produce fewer contradictions and triple omissions than fine-tuned models, and these errors increase more rapidly with graph size for fine-tuned models, highlighting differences between the two approaches that reference-based metrics may not fully capture.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Benchmark (surveying), Graph, Empirical research, Artificial intelligence, Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange