Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?

L. Gao, J. Tow, S. Biderman, S. others Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, K. McDonell, N. Muennighoff, J. Phang, L. Reynolds, E. Tang, A. Thite, B. Wang, K. Wang, A. Zou, A framework for few-shot language model evaluation, 2021. doi: 10.5281/zenodo.5371628

2021

Earlier work this paper cites.

Sparks of artificial general intelligence: Early experiments with gpt-4 (2023). arXiv:2303.12712

Original

S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, Y. Zhang, · 2023

Earlier work this paper cites.

Knowledge graphs and their role in the knowledge engineering of the 21st century (dagstuhl seminar 22372) (2023). URL: https://www.dagstuhl.de/en/seminars/seminar-calendar/seminar-details/22372 . doi: 10.4230/DAGREP.12.9.60

P. Groth, E. Simperl, M. van Erp, D. Vrandečić, · 2023

Earlier work this paper cites.

Unifying large language models and knowledge graphs: A roadmap (2023). arXiv:2306.08302

Original

S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, X. Wu, · 2023

Earlier work this paper cites.

L.-P. Meyer, C. Stadler, J. Frey, N. Radtke, K. Junghanns, R. Meissner, G. Dziwis, K. Bulert, M. Martin, Llm-assisted knowledge graph engineering: Experiments with chatgpt, 2023a. arXiv:2307.06917 , to appear in conference proceedings of AI-Tomorrow-23, 29.+30. 6.2023 in Leipzig, Germany

Original

2023

Cited alongside, same era.

L.-P. Meyer, J. Frey, K. Junghanns, F. Brei, K. Bulert, S. Gründer-Fahrer, M. Martin, Developing a scalable benchmark for assessing large language models in knowledge graph engineering, 2023b. arXiv:2308.16622 , to appear in poster proceedings of Semantics-23, 20-22. 9. 2023, Leipzig, Germany

Original

2023

Cited alongside, same era.

Enhancing knowledge graph construction using large language models (2023). arXiv:2305.04676

Original

M. Trajanoska, R. Stojanov, D. Trajanov, · 2023

Cited alongside, same era.

Iterative zero-shot llm prompting for knowledge graph construction (2023). arXiv:2307.01128

Original

S. Carta, A. Giuliani, L. Piano, A. S. Podda, L. Pompianu, S. G. Tiddia, · 2023

Cited alongside, same era.

Y. Zhu, X. Wang, J. Chen, S. Qiao, Y. Ou, Y. Yao, S. Deng, H. Chen, N. Zhang, Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities, 2023. arXiv:2305.13168

Original

2023

Closest in time.

Gpt4graph: Can large language models understand graph structured data ? an empirical evaluation and benchmarking (2023). arXiv:2305.15066

Original

J. Guo, L. Du, H. Liu, M. Zhou, X. He, S. Han, · 2023

Closest in time.

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,

Original

A. Srivastava, et al., · 2023

Closest in time.

L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, I. Stoica, Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. arXiv:2306.05685

Original

2023

Closest in time.