Prompting as Probing: Using Language Models for Knowledge Base Construction

Language models are few-shot learners,

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., · 1901

Earlier work this paper cites.

L. Yao, C. Mao, Y. Luo, Kg-bert: Bert for knowledge graph completion, 2019. arXiv:1909.03193

Original

1909

Earlier work this paper cites.

BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA 0 (2019). arXiv:1911.03681

Original

N. Poerner, U. Waltinger, H. Schütze, · 1911

Earlier work this paper cites.

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts (2020) 4222–4235. arXiv:2010.15980

Original

T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, S. Singh, · 2010

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding,

Original

J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, · 2018

Earlier work this paper cites.

Finbert: Financial sentiment analysis with pre-trained language models,

Original

D. Araci, · 2019

Earlier work this paper cites.

Language Models as Knowledge Bases?,

F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y. Wu, A. Miller, · 2019

Earlier work this paper cites.

Roberta: A robustly optimized bert pretraining approach,

Original

Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, · 2019

Earlier work this paper cites.

ProtTrans: Towards cracking the language of life’s code through self-supervised deep learning and high performance computing,

Original

A. Elnaggar, M. Heinzinger, C. Dallago, G. Rihawi, Y. Wang, et al., · 2020

Earlier work this paper cites.

Approximate knowledge graph query answering: from ranking to binary classification,

R. v. Bakel, T. Aleksiev, D. Daza, D. Alivanistos, M. Cochez, · 2020

Earlier work this paper cites.

How Can We Know What Language Models Know?,

Z. Jiang, F. F. Xu, J. Araki, G. Neubig, · 2020

Cited alongside, same era.

Inducing Relational Knowledge from BERT,

Z. Bouraoui, J. Camacho-Collados, S. Schockaert, · 2020

Cited alongside, same era.

KnowlyBERT - Hybrid Query Answering over Language Models and Knowledge Graphs,

J.-C. Kalo, L. Fichtel, P. Ehler, W.-T. Balke, · 2020

Cited alongside, same era.

Inductive entity representations from text via link prediction,

D. Daza, M. Cochez, P. Groth, · 2021

Cited alongside, same era.

Learning how to ask: Querying LMs with mixtures of soft prompts,

G. Qin, J. Eisner, · 2021

Cited alongside, same era.

Language Models As or For Knowledge Bases (2021). arXiv:2110.04888

Original

S. Razniewski, A. Yates, N. Kassner, G. Weikum, · 2021

Time-Aware Language Models as Temporal Knowledge Bases (2021). arXiv:2106.15110

Original

B. Dhingra, J. R. Cole, J. M. Eisenschlos, D. Gillick, J. Eisenstein, W. W. Cohen, · 2021

Later among the works it cites.

Can Language Models be Biomedical Knowledge Bases? (2021) 4723–4734. arXiv:2109.07154

Original

M. Sung, J. Lee, S. Yi, M. Jeon, S. Kim, J. Kang, · 2021

Later among the works it cites.

Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models (2021). arXiv:2110.08173

Original

Z. Meng, F. Liu, E. Shareghi, Y. Su, C. Collins, N. Collier, · 2021

Later among the works it cites.

Do Judge an Entity by Its Name! Entity Typing Using Language Models,

R. Biswas, R. Sofronova, M. Alam, N. Heist, H. Paulheim, H. Sack, · 2021

Later among the works it cites.

Prompt programming for large language models: Beyond the few-shot paradigm,