Fetching the paper…

Reading the bibliography…

2024

Citekit: A Modular Toolkit for Large Language Model Citation Generation

Shen, Jiajun, Zhou, Tong, Chen, Yubo et al.

Original
PDF
Post on X

Understand

Enabling Large Language Models (LLMs) to generate citations in Question-Answering (QA) tasks is an emerging paradigm aimed at enhancing the verifiability of their responses when LLMs are utilizing external references to generate an answer.

However, there is currently no unified framework to standardize and fairly compare different citation generation methods, leading to difficulties in reproducing different methods and a comprehensive assessment.
To cope with the problems above, we introduce \name, an open-source and modular toolkit designed to facilitate the implementation and evaluation of existing citation generation methods, while also fostering the development of new approaches to improve citation quality in LLM outputs.
This tool is highly extensible, allowing users to utilize 4 main modules and 14 components to construct a pipeline, evaluating an existing method or innovative designs.

Citekit: A Modular Toolkit for Large Language Model Citation Generation · Around

Built on

Dense passage retrieval for open-domain question answering
Original
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020 · 2004
Earlier work this paper cites.
Retrieval-augmented generation for knowledge-intensive nlp tasks
Original
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021 · 2005
Earlier work this paper cites.
Hotpotqa: A dataset for diverse, explainable multi-hop question answering
Original
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018 · 2018
Earlier work this paper cites.
CommonsenseQA: A question answering challenge targeting commonsense knowledge
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019 · 2019
Earlier work this paper cites.
Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020 · 2020
Earlier work this paper cites.
AmbigQA: Answering ambiguous open-domain questions
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020 · 2020
Earlier work this paper cites.
TRUE: Re-evaluating factual consistency evaluation
Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022 · 2022
Earlier work this paper cites.
Self-rag: Learning to retrieve, generate, and critique through self-reflection
Original
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023 · 2023
Earlier work this paper cites.
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions
Original
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023 · 2023
Earlier work this paper cites.
1-pager: One pass answer generation and evidence retrieval
Original
Palak Jain, Livio Baldini Soares, and Tom Kwiatkowski. 2023 · 2023
Earlier work this paper cites.

Similar

Evaluating open-domain question answering in the era of large language models
Original
Ehsan Kamalloo, Nouha Dziri, Charles L. A. Clarke, and Davood Rafiei. 2023 · 2023
Cited alongside, same era.
Towards reliable and fluent large language models: Incorporating feedback learning loops in qa systems
Original
Dongyub Lee, Taesun Whang, Chanhee Lee, and Heuiseok Lim. 2023 · 2023
Cited alongside, same era.
A survey of large language models attribution
Original
Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, and Min Zhang. 2023 · 2023
Cited alongside, same era.
Asqa: Factoid questions meet long-form answers
Original
Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2023 · 2023
Cited alongside, same era.
Recitation-augmented language models

Then

Retrieval-augmented generation for large language models: A survey
Original
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024 · 2024
Closest in time.
Ask, assess, and refine: Rectifying factual consistency and hallucination in LLMs with metric-guided feedback learning
Dongyub Lee, Eunhwan Park, Hodong Lee, and Heuiseok Lim. 2024 · 2024
Closest in time.
Gpt-4 technical report
Original
OpenAI. 2024 · 2024
Closest in time.
On the capacity of citation generation by large language models
Original
Haosheng Qian, Yixing Fan, Ruqing Zhang, and Jiafeng Guo. 2024 · 2024
Closest in time.
Attribute first, then generate: Locally-attributable grounded text generation
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2023 · 2023

Cited alongside, same era.

Evaluation of attribution bias in retrieval-augmented large language models

Amin Abolghasemi, Leif Azzopardi, Seyyed Hadi Hashemi, Maarten de Rijke, and Suzan Verberne. 2024 · 2024

Cited alongside, same era.

Llama 3 model card

AI@Meta. 2024 · 2024

Cited alongside, same era.

Localizing factual inconsistencies in attributable text generation

Arie Cattan, Paul Roit, Shiyue Zhang, David Wan, Roee Aharoni, Idan Szpektor, Mohit Bansal, and Ido Dagan. 2024 · 2024

Cited alongside, same era.

Learning to plan and generate text with citations

Constanza Fierro, Reinald Kim Amplayo, Fantine Huot, Nicola De Cao, Joshua Maynez, Shashi Narayan, and Mirella Lapata. 2024 · 2024

Cited alongside, same era.

RARR: Researching and revising what language models say, using language models

Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023a

Cited in the paper.

Enabling large language models to generate text with citations

Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023b

Cited in the paper.

Training language models to generate text with citations via fine-grained rewards

Original

Chengyu Huang, Zeqiu Wu, Yushi Hu, and Wenya Wang. 2024a

Cited in the paper.

Aviv Slobodkin, Eran Hirsch, Arie Cattan, Tal Schuster, and Ido Dagan. 2024 · 2024

Closest in time.

Towards verifiable text generation with evolving memory and self-reflection

Hao Sun, Hengyi Cai, Bo Wang, Yingyan Hou, Xiaochi Wei, Shuaiqiang Wang, Yan Zhang, and Dawei Yin. 2024 · 2024

Closest in time.

Atomic fact decomposition helps attributed question answering

Original

Zhichao Yan, Jiapu Wang, Jiaoyan Chen, Xiaoli Li, Ru Li, and Jeff Z. Pan. 2024 · 2024

Closest in time.

Effective large language model adaptation for improved grounding and citation generation

Xi Ye, Ruoxi Sun, Sercan Ö. Arik, and Tomas Pfister. 2024 · 2024

Closest in time.

Verifiable by design: Aligning language models to quote from pre-training data

Jingyu Zhang, Marc Marone, Tianjian Li, Benjamin Van Durme, and Daniel Khashabi. 2024 · 2024

Closest in time.