UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models

Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems , Vol. 33. Curran Associates, Inc., 1877–1901

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 1901

Earlier work this paper cites.

Long short-term memory

Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997

Earlier work this paper cites.

An information-theoretic perspective of tf–idf measures

Akiko Aizawa. 2003 · 2003

Earlier work this paper cites.

Multimodal Named Entity Disambiguation for Noisy Social Media Posts. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, Melbourne, Australia, 2000–2008

Seungwhan Moon, Leonardo Neves, and Vitor Carvalho. 2018 · 2008

Earlier work this paper cites.

Efficient Estimation of Word Representations in Vector Space. In International Conference on Learning Representations

Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013 · 2013

Earlier work this paper cites.

Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, San Diego, California, 1256–1261

Matthew Francis-Landau, Greg Durrett, and Dan Klein. 2016 · 2016

Earlier work this paper cites.

Hierarchical question-image co-attention for visual question answering. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS’16) . Curran Associates Inc., Red Hook, NY, USA, 289–297

Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016 · 2016

Earlier work this paper cites.

Named entity disambiguation for noisy text

Original

Yotam Eshel, Noam Cohen, Kira Radinsky, Shaul Markovitch, Ikuya Yamada, and Omer Levy. 2017 · 2017

Earlier work this paper cites.

Deep Joint Entity Disambiguation with Local Neural Attention. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Copenhagen, Denmark, 2619–2629

Octavian-Eugen Ganea and Thomas Hofmann. 2017 · 2017

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding

Original

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Earlier work this paper cites.

Decoupled Weight Decay Regularization. In International Conference on Learning Representations

Ilya Loshchilov and Frank Hutter. 2018 · 2018

Earlier work this paper cites.

Question Answering by Reasoning Across Documents with Graph Convolutional Networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Association for Computational Linguistics, Minneapolis, Minnesota, 2306–2317

Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019 · 2019

Earlier work this paper cites.

Learning Dense Representations for Entity Retrieval. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL) . Association for Computational Linguistics, Hong Kong, China, 528–537

Daniel Gillick, Sayali Kulkarni, Larry Lansing, Alessandro Presta, Jason Baldridge, Eugene Ie, and Diego Garcia-Olano. 2019 · 2019

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT , Vol. 1. 2

Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019

Earlier work this paper cites.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019 · 2019

Earlier work this paper cites.

Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Florence, Italy, 4258–4264

Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2019 · 2019

Earlier work this paper cites.

Richpedia: a large-scale, comprehensive multi-modal knowledge graph

Meng Wang, Haofen Wang, Guilin Qi, and Qiushuo Zheng. 2020 · 2020

Earlier work this paper cites.

Scalable Zero-shot Entity Linking with Dense Entity Retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 6397–6407

Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020 · 2020

Earlier work this paper cites.

Autoregressive Entity Retrieval. In ICLR 2021-9th International Conference on Learning Representations , Vol. 2021. ICLR

N De Cao, G Izacard, S Riedel, and F Petroni. 2020 · 2021

Earlier work this paper cites.

Multimodal Entity Linking: A New Dataset and A Baseline. In Proceedings of the 29th ACM International Conference on Multimedia (Virtual Event, China) (MM ’21) . Association for Computing Machinery, New York, NY, USA, 993–1001

Jingru Gan, Jinchang Luo, Haiwei Wang, Shuhui Wang, Wei He, and Qingming Huang. 2021 · 2021

Earlier work this paper cites.

Entity-Based Knowledge Conflicts in Question Answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 7052–7063

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021 · 2021

Earlier work this paper cites.

Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (Proceedings of Machine Learning Research, Vol. 139) . PMLR, 8748–8763

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021 · 2021

Earlier work this paper cites.

Attention-Based Multimodal Entity Linking with High-Quality Images. In Database Systems for Advanced Applications: 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11–14, 2021, Proceedings, Part II (Taipei, Taiwan). Springer-Verlag, Berlin, Heidelberg, 533–548

Li Zhang, Zhixu Li, and Qiang Yang. 2021 · 2021

Cited alongside, same era.

Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning . PMLR, 12697–12706

Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021 · 2021

Cited alongside, same era.

Weibo-mel, Wikidata-mel and Richpedia-mel: multimodal entity linking benchmark datasets. In Knowledge Graph and Semantic Computing: Knowledge Graph Empowers New Infrastructure Construction: 6th China Conference, CCKS 2021, Guangzhou, China, November 4-7, 2021, Proceedings 6 . Springer, 315–320

Xingchen Zhou, Peng Wang, Guozheng Li, Jiafeng Xie, and Jiangheng Wu. 2021 · 2021

Cited alongside, same era.

Multimodal Knowledge Learning for Named Entity Disambiguation. In Findings of the Association for Computational Linguistics: EMNLP 2022 . Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 3160–3169

Generative multimodal entity linking

Original

Senbao Shi, Zhenran Xu, Baotian Hu, and Min Zhang. 2023 · 2023

Later among the works it cites.

Llama: Open and efficient foundation language models

Original

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023

Later among the works it cites.

Benchmarking Diverse-Modal Entity Linking with Generative Models. In Findings of the Association for Computational Linguistics: ACL 2023 . 7841–7857

Sijia Wang, Alexander Hanbo Li, Henghui Zhu, Sheng Zhang, Pramuditha Perera, Chung-Wei Hang, Jie Ma, William Yang Wang, Zhiguo Wang, Vittorio Castelli, et al · 2023

Later among the works it cites.