Fetching the paper…

Reading the bibliography…

2024

Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge

Delile, Julien, Mukherjee, Srayanta, Van Pamel, Anton et al.

Original
PDF
Post on X

Understand

Large language models (LLMs) are transforming the way information is retrieved with vast amounts of knowledge being summarized and presented via natural language conversations.

Yet, LLMs are prone to highlight the most frequently seen pieces of information from the training set and to neglect the rare ones.
In the field of biomedical research, latest discoveries are key to academic and industrial actors and are obscured by the abundance of an ever-increasing literature corpus (the information overload problem).
Surfacing new associations between biomedical entities, e.g., drugs, genes, diseases, with LLMs becomes a challenge of capturing the long-tail knowledge of the biomedical scientific production.

Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge · Around

Built on

Scispacy: Fast and robust models for biomedical natural language processing
Original
Neumann M, King D, Beltagy I, Ammar W. 2019 · 1902
Earlier work this paper cites.
Ernie: Enhanced language representation with informative entities
Original
Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q. 2019 · 1905
Earlier work this paper cites.
Comet: Commonsense transformers for automatic knowledge graph construction
Original
Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y. 2019 · 1906
Earlier work this paper cites.
Barack’s wife hillary: Using knowledge-graphs for fact-aware language modeling
Original
Logan RL, Liu NF, Peters ME, Gardner M, Singh S. 2019 · 1906
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach . URL https://arxiv.org/abs/1907.11692v1
Original
Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V, Allen PG. 2019 · 1907
Earlier work this paper cites.
Kagnet: Knowledge-aware graph networks for commonsense reasoning
Original
Lin BY, Chen X, Chen J, Ren X. 2019 · 1909
Earlier work this paper cites.
Kg-bert: Bert for knowledge graph completion . URL https://arxiv.org/abs/1909.03193v2
Original
Yao L, Mao C, Luo Y. 2019 · 1909
Earlier work this paper cites.
Long short-term memory
Hochreiter S, Schmidhuber J. 1997 · 1997
Earlier work this paper cites.
The nlm indexing initiative
Aronson AR, Bodenreider O, Chang HF, Humphrey SM, Mork JG, Nelson SJ, Rindflesch TC, Wilbur WJ. 2000 · 2000
Earlier work this paper cites.
Realm: Retrieval-augmented language model pre-training
Original
Guu K, Lee K, Tung Z, Pasupat P, Chang MW. 2020 · 2002
Earlier work this paper cites.
Colbert: Efficient and effective passage search via contextualized late interaction over bert
Original
Khattab O, Zaharia M. 2020 · 2004
Earlier work this paper cites.
Scalable multi-hop relational reasoning for knowledge-aware question answering
Original
Feng Y, Chen X, Lin BY, Wang P, Yan J, Ren X. 2020 · 2005
Earlier work this paper cites.
Retrieval-augmented generation for knowledge-intensive nlp tasks
Original
Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih WT, Rocktäschel T, Riedel S, Kiela D. 2020 · 2005
Earlier work this paper cites.

Similar

Domain-specific language model pretraining for biomedical natural language processing
Original
Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, Naumann T, Gao J, Poon H. 2020 · 2007
Cited alongside, same era.
Investigating pretrained language models for graph-to-text generation
Original
Ribeiro LF, Schmitt M, Schütze H, Gurevych I. 2020 · 2007
Cited alongside, same era.
Semi-supervised classification with graph convolutional networks
Original
Kipf TN, Welling M. 2016 · 2017
Cited alongside, same era.
Commonsense knowledge aware conversation generation with graph attention
Zhou H, Young T, Huang M, Zhao H, Xu J, Zhu X. 2018 · 2018
Cited alongside, same era.
Pretrain-kge: Learning knowledge representation from pretrained language models
Zhang Z, Liu X, Zhang Y, Su Q, Sun X, He B. 2020 · 2020

Then

A survey on efficient processing of similarity queries over neural embeddings . URL https://arxiv.org/abs/2204.07922v1
Original
Wang Y. 2022 · 2022
Later among the works it cites.
From discrimination to generation: Knowledge graph completion with generative transformer
Original
Xie X, Zhang N, Li Z, Deng S, Chen H, Xiong F, Chen M, Chen H. 2022 · 2022
Later among the works it cites.
Biomedical ner for the enterprise with distillated bern2 and the kazu framework . URL https://arxiv.org/abs/2212.00223v1
Original
Yoon W, Jackson R, Ford E, Poroshin V, Kang J. 2022 · 2022
Later among the works it cites.
Survey of query-based text summarization . URL https://arxiv.org/abs/2211.11548v1
Original
Yu H. 2022 · 2022
Later among the works it cites.
Greaselm: Graph reasoning enhanced language models for question answering

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Jointgt: Graph-text joint representation learning for text generation from knowledge graphs

Ke P, Ji H, Ran Y, Cui X, Wang L, Song L, Zhu X, Huang M. 2021 · 2021

Cited alongside, same era.

Qa-gnn: Reasoning with language models and knowledge graphs for question answering

Yasunaga M, Ren H, Bosselut A, Liang P, Leskovec J. 2021 · 2021

Cited alongside, same era.

Langchain. URL https://github.com/hwchase17/langchain

Chase H. 2022 · 2022

Cited alongside, same era.

Rho ( ρ \rho ): Reducing hallucination in open-domain dialogues with knowledge grounding . URL https://arxiv.org/abs/2212.01588v2

Ji Z, Liu Z, Lee N, Yu T, Wilie B, Zeng M, Fung P. 2022 · 2022

Cited alongside, same era.

Llamaindex. URL https://github.com/jerryjliu/gpt_index

Liu J. 2022 · 2022

Cited alongside, same era.

Biored: a rich biomedical relation extraction dataset

Luo L, Lai PT, Wei CH, Arighi CN, Lu Z. 2022 · 2022

Cited alongside, same era.

Jointlk: Joint reasoning with language models and knowledge graphs for commonsense question answering

Sun Y, Shi Q, Qi L, Zhang Y. 2021 · 2022

Cited alongside, same era.

Zhang X, Bosselut A, Yasunaga M, Ren H, Liang P, Manning CD, Leskovec J. 2022 · 2022

Later among the works it cites.

Factkb: Generalizable factuality evaluation using language models enhanced with factual knowledge . URL https://arxiv.org/abs/2305.08281v1

Original

Feng S, Balachandran V, Bai Y, Tsvetkov Y. 2023 · 2023

Later among the works it cites.

Pive: Prompting with iterative verification improving graph-based generative capability of llms . URL https://arxiv.org/abs/2305.12392v1

Han J, Collier N, Buntine W, Shareghi E. 2023 · 2023

Later among the works it cites.

Lost in the middle: How language models use long contexts . URL https://arxiv.org/abs/2307.03172v2

Liu NF, Lin K, Hewitt J, Paranjape A, Bevilacqua M, Petroni F, Liang P. 2023 · 2023

Later among the works it cites.

Unifying large language models and knowledge graphs: A roadmap . URL https://arxiv.org/abs/2306.08302v1

Pan S, Member S, Luo L, Wang Y, Chen C, Wang J, Wu X. 2023 · 2023

Later among the works it cites.

The current state of summarization . URL https://arxiv.org/abs/2305.04853v2

Retkowski F. 2023 · 2023

Later among the works it cites.

Exploring the limits of chatgpt for query or aspect-based text summarization . URL http://arxiv.org/abs/2302.08081

Yang X, Li Y, Zhang X, Chen H, Cheng W. 2023 · 2023

Later among the works it cites.

Megabyte: Predicting million-byte sequences with multiscale transformers . URL https://arxiv.org/abs/2305.07185v2

Yu L, Simig D, Flaherty C, Aghajanyan A, Zettlemoyer L, Lewis M. 2023 · 2023

Later among the works it cites.

Benchmarking large language models for news summarization . URL https://arxiv.org/abs/2301.13848v1

Zhang T, Ladhak F, Durmus E, Liang P, McKeown K, Hashimoto TB. 2023 · 2023

Later among the works it cites.