Structure-aware Domain Knowledge Injection for Large Language Models

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Bertscore: Evaluating text generation with bert

Original

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020 · 1904

Earlier work this paper cites.

Barack’s wife hillary: Using knowledge-graphs for fact-aware language modeling

Original

Robert L Logan IV, Nelson F Liu, Matthew E Peters, Matt Gardner, and Sameer Singh. 2019 · 1906

Earlier work this paper cites.

Ctrl: A conditional transformer language model for controllable generation

Original

Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019 · 1909

Earlier work this paper cites.

A neural probabilistic language model

Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. 2000 · 2000

Earlier work this paper cites.

A revision of bloom’s taxonomy: An overview

David R Krathwohl. 2002 · 2002

Earlier work this paper cites.

Don’t stop pretraining: Adapt language models to domains and tasks

Original

Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020 · 2004

Earlier work this paper cites.

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

What disease does this patient have? a large-scale open domain question answering dataset from medical exams

Original

Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020 · 2009

Earlier work this paper cites.

Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps

Original

Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020 · 2011

Earlier work this paper cites.

Dureader: a chinese machine reading comprehension dataset from real-world applications

Original

Wei He, Kai Liu, Jing Liu, Yajuan Lyu, Shiqi Zhao, Xinyan Xiao, Yuan Liu, Yizhong Wang, Hua Wu, Qiaoqiao She, et al. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Hotpotqa: A dataset for diverse, explainable multi-hop question answering

Original

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018 · 2018

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Earlier work this paper cites.

K-bert: Enabling language representation with knowledge graph

Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. 2020 · 2020

Earlier work this paper cites.

Ernie 2.0: A continual pre-training framework for language understanding

Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020 · 2020

Earlier work this paper cites.

A dataset of information-seeking questions and answers anchored in research papers

Original

Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. 2021 · 2021

Earlier work this paper cites.

Domain-specific language model pretraining for biomedical natural language processing

Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021 · 2021

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Cited alongside, same era.

Musique: Multihop questions via single-hop question composition

Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022 · 2022

Cited alongside, same era.

Dkplm: decomposable knowledge-enhanced pre-trained language model for natural language understanding

Taolin Zhang, Chengyu Wang, Nan Hu, Minghui Qiu, Chengguang Tang, Xiaofeng He, and Jun Huang. 2022 · 2022

Cited alongside, same era.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Cited alongside, same era.

Understanding retrieval augmentation for long-form question answering

A survey of large language models

Original

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023 · 2023

Later among the works it cites.

Introducing meta llama 3: The most capable openly available llm to date

Meta AI. 2024 · 2024

Closest in time.

Deepseek llm: Scaling open-source language models with longtermism

Original

Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024 · 2024

Closest in time.

Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation

Original

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024 · 2024

Closest in time.

Multi-view content-aware indexing for long document retrieval