Fetching the paper…
Reading the bibliography…
Multimodal document retrieval aims to identify and retrieve various forms of multimodal content, such as figures, tables, charts, and layout information from extensive documents.
Extended boolean information retrieval
Gerard Salton, Edward A. Fox, and Harry Wu. 1983 · 1983
Earlier work this paper cites.
Okapi at TREC-3
Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994 · 1994
Earlier work this paper cites.
Optical character recognition
Shunji Mori, Hirobumi Nishida, and Hiromitsu Yamada. 1999 · 1999
Earlier work this paper cites.
Layout and content extraction for PDF documents
Hui Chao and Jian Fan. 2004 · 2004
Earlier work this paper cites.
An overview of the tesseract ocr engine
Ray Smith. 2007 · 2007
Earlier work this paper cites.
Multi-modal information integration for document retrieval
Ehtesham Hassan, Santanu Chaudhury, and Madan Gopal. 2013 · 2013
Earlier work this paper cites.
A survey of modern optical character recognition techniques
Eugene Borovikov. 2014 · 2014
Earlier work this paper cites.
Image captioning with semantic attention
Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. 2016 · 2016
Earlier work this paper cites.
Optical Character Recognition Systems , pages 9–41
Arindam Chaudhuri, Krupa Mandaviya, Pratixa Badelia, and Soumya K. Ghosh. 2017 · 2017
Earlier work this paper cites.
Convolutional image captioning
Jyoti Aneja, Aditya Deshpande, and Alexander G. Schwing. 2018 · 2018
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Natural questions: A benchmark for question answering research
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019 · 2019
Earlier work this paper cites.
Dense passage retrieval for open-domain question answering
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020
Earlier work this paper cites.
Colbert: Efficient and effective passage search via contextualized late interaction over bert
Omar Khattab and Matei Zaharia. 2020 · 2020
Earlier work this paper cites.
Monolingual and multilingual reduction of gender bias in contextualized representations
Sheng Liang, Philipp Dufter, and Hinrich Schütze. 2020 · 2020
Earlier work this paper cites.
Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020 · 2020
Earlier work this paper cites.
Layoutlm: Pre-training of text and layout for document image understanding
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020 · 2020
Earlier work this paper cites.
Document ai: Benchmarks, models and applications
Lei Cui, Yiheng Xu, Tengchao Lv, and Furu Wei. 2021 · 2021
Earlier work this paper cites.
DocOIE: A document-level context-aware dataset for OpenIE
Kuicai Dong, Zhao Yilin, Aixin Sun, Jung-Jae Kim, and Xiaoli Li. 2021 · 2021
Earlier work this paper cites.
CUAD: an expert-annotated NLP dataset for legal contract review
Dan Hendrycks, Collin Burns, Anya Chen, and Spencer Ball. 2021 · 2021
Earlier work this paper cites.
Docvqa: A dataset for VQA on document images
Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2021 · 2021
Earlier work this paper cites.
Document collection visual question answering
Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. 2021 · 2021
Earlier work this paper cites.
Layoutlmv2: Multi-modal pre-training for visually-rich document understanding
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei A. F. Florêncio, Cha Zhang, Wanxiang Che, Min Zhang, and Lidong Zhou. 2021 · 2021
Earlier work this paper cites.
A generative model for end-to-end argument mining with reconstructed positional encoding and constrained pointer mechanism
Jianzhu Bao, Yuhang He, Yang Sun, Bin Liang, Jiachen Du, Bing Qin, Min Yang, and Ruifeng Xu. 2022a · 2022
Earlier work this paper cites.
AEG: Argumentative essay generation via a dual-decoder model with content planning
Jianzhu Bao, Yasheng Wang, Yitong Li, Fei Mi, and Ruifeng Xu. 2022b · 2022
Cited alongside, same era.
Syntactic multi-view learning for open information extraction
Kuicai Dong, Aixin Sun, Jung-Jae Kim, and Xiaoli Li. 2022 · 2022
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 · 2022
Cited alongside, same era.
Unsupervised dense information retrieval with contrastive learning
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022 · 2022
Cited alongside, same era.
Infographicvqa
Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V. Jawahar. 2022 · 2022
Cited alongside, same era.
Text embeddings by weakly-supervised contrastive pre-training
Flashattention-2: Faster attention with better parallelism and work partitioning
Tri Dao. 2024 · 2024
Later among the works it cites.
MMVQA: A comprehensive dataset for investigating multipage multimodal information retrieval in pdf-based visual question answering
Yihao Ding, Kaixuan Ren, Jiabin Huang, Siwen Luo, and Soyeon Caren Han. 2024 · 2024
Later among the works it cites.
MC-indexing: Effective long document retrieval via multi-view content-aware indexing
Kuicai Dong, Derrick Goh Xin Deik, Yi Quan Lee, Hao Zhang, Xiangyang Li, Cong Zhang, and Yong Liu. 2024 · 2024
Later among the works it cites.
Colpali: Efficient document retrieval with vision language models
Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2024 · 2024
Later among the works it cites.
Building and better understanding vision-language models: insights and future directions
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022 · 2022
Cited alongside, same era.
Multi-view document representation learning for open-domain dense retrieval
Shunyu Zhang, Yaobo Liang, Ming Gong, Daxin Jiang, and Nan Duan. 2022 · 2022
Cited alongside, same era.
Towards complex document understanding by discrete reasoning
Fengbin Zhu, Wenqiang Lei, Fuli Feng, Chao Wang, Haozhou Zhang, and Tat-Seng Chua. 2022 · 2022
Cited alongside, same era.
Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 · 2023
Cited alongside, same era.
A synthetic data generation framework for grounded dialogues
Jianzhu Bao, Rui Wang, Yasheng Wang, Aixin Sun, Yitong Li, Fei Mi, and Ruifeng Xu. 2023 · 2023
Cited alongside, same era.
Walking down the memory maze: Beyond context limit through interactive reading
Howard Chen, Ramakanth Pasunuru, Jason Weston, and Asli Celikyilmaz. 2023 · 2023
Cited alongside, same era.
From speculation detection to trustworthy relational tuples in information extraction
Kuicai Dong, Aixin Sun, Jung-jae Kim, and Xiaoli Li. 2023a · 2023
Cited alongside, same era.
Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. 2024 · 2024
Later among the works it cites.
Unified multimodal interleaved document representation for retrieval
Jaewoo Lee, Joonho Ko, Jinheon Baek, Soyeong Jeong, and Sung Ju Hwang. 2024 · 2024
Later among the works it cites.
Multimodal ArXiv: A dataset for improving scientific comprehension of large vision-language models
Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. 2024 · 2024
Later among the works it cites.
Unifying multimodal retrieval via document screenshot embedding
Xueguang Ma, Sheng-Chieh Lin, Minghan Li, Wenhu Chen, and Jimmy Lin. 2024a · 2024
Later among the works it cites.
MMLONGBENCH-DOC: benchmarking long-context document understanding with visualizations
Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, Pan Zhang, Liangming Pan, Yu-Gang Jiang, Jiaqi Wang, Yixin Cao, and Aixin Sun. 2024b · 2024
Later among the works it cites.
Hello gpt-4o: We’re announcing gpt-4o, our new flagship model that can reason across audio, vision, and text in real time
OpenAI. 2024 · 2024
Later among the works it cites.
Introducing qwen-vl
Qwen-Team. 2024 · 2024
Later among the works it cites.
Question-based retrieval using atomic units for enterprise rag
Vatsal Raina and Mark Gales. 2024 · 2024
Later among the works it cites.
Yuwei Wan, Yixuan Liu, Aswathy Ajith, Clara Grazian, Bram Hoex, Wenjie Zhang, Chunyu Kit, Tong Xie, and Ian Foster. 2024 · 2024
Later among the works it cites.
Mineru: An open-source solution for precise document content extraction
Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. 2024 · 2024
Later among the works it cites.
SciMMIR: Benchmarking scientific multi-modal information retrieval
Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, and Chenghua Lin. 2024 · 2024
Later among the works it cites.
Docbench: A benchmark for evaluating llm-based document reading systems
Anni Zou, Wenhao Yu, Hongming Zhang, Kaixin Ma, Deng Cai, Zhuosheng Zhang, Hai Zhao, and Dong Yu. 2024 · 2024
Later among the works it cites.
Copyspec: Accelerating llms with speculative copy-and-paste without compromising quality
Razvan-Gabriel Dumitru, Minglai Yang, Vikas Yadav, and Mihai Surdeanu. 2025 · 2025
Closest in time.
Improving the data-efficiency of reinforcement learning by warm-starting with llm
Thang Duong, Minglai Yang, and Chicheng Zhang. 2025 · 2025
Closest in time.
CoIR: A comprehensive benchmark for code information retrieval models
Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Hao Zhang, Xinyi Dai, Yasheng Wang, and Ruiming Tang. 2025 · 2025
Closest in time.
Adaptive schema-aware event extraction with retrieval-augmented generation
Sheng Liang, Hang Lv, Zhihao Wen, Yaxiong Wu, Yongyue Zhang, Hao Wang, and Yong Liu. 2025 · 2025
Closest in time.
From human memory to ai memory: A survey on memory mechanisms in the era of llms
Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. 2025 · 2025
Closest in time.
How is llm reasoning distracted by irrelevant context? an analysis using a controlled benchmark
Minglai Yang, Ethan Huang, Liang Zhang, Mihai Surdeanu, William Wang, and Liangming Pan. 2025 · 2025
Closest in time.
Cross-modal information flow in multimodal large language models
Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. 2025 · 2025
Closest in time.