Fetching the paper…
Reading the bibliography…
Vision-Language Models (VLMs) have shown strong capabilities in document understanding, particularly in identifying and extracting textual information from complex documents.
Towards VQA models that can read
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach · 2019
Earlier work this paper cites.
Docvqa: A dataset for VQA on document images
Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar · 2021
Earlier work this paper cites.
Layoutlmv3: pre-training for document ai with unified text and image masking, 2022
Y. Huang, T. Lv, L. Cui, Y. Lu, and F. Wei · 2022
Earlier work this paper cites.
Ocr-free document understanding transformer
Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park · 2022
Earlier work this paper cites.
ChartQA: A benchmark for question answering about charts with visual and logical reasoning
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque · 2022
Earlier work this paper cites.
Document understanding dataset and evaluation (DUDE)
Jordy Van Landeghem, Rafal Powalski, Rubèn Tito, Dawid Jurkiewicz, Matthew B. Blaschko, Lukasz Borchmann, Mickaël Coustaty, Sien Moens, Michal Pietruszka, Bertrand Anckaert, Tomasz Stanislawek, Pawel Józiak, and Ernest Valveny · 2023
Earlier work this paper cites.
Pix2Struct: Screenshot parsing as pretraining for visual language understanding
Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova · 2023
Earlier work this paper cites.
Vision-language models for vision tasks: A survey
Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu · 2023
Earlier work this paper cites.
Docformerv2: Local features for document understanding
Srikar Appalaraju, Peng Tang, Qi Dong, Nishant Sankaran, Yichu Zhou, and R. Manmatha · 2024
Earlier work this paper cites.
Learning to ground vlms without forgetting
Aritra Bhowmik, Mohammad Mahdi Derakhshani, Dennis C. Koelma, Martin R. Oswald, Yuki M. Asano, and Cees G. M. Snoek · 2024
Earlier work this paper cites.
PIN: positional insert unlocks object localisation abilities in vlms
Michael Dorkenwald, Nimrod Barazani, Cees G. M. Snoek, and Yuki M. Asano · 2024
Earlier work this paper cites.
Building and better understanding vision-language models: insights and future directions
Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon · 2024
Earlier work this paper cites.
A bounding box is worth one token: interleaving layout and text in a large language model for document understanding, 2024
J. Lu, H. Yu, Y. Wang, Y. Ye, J. Tang, Z. Yang, B. Wu, Q. Liu, H. Feng, H. Wang, H. Liu, and C. Huang · 2024
Cited alongside, same era.
Ahmad Mohammadshirazi, Pinaki Prasad Guha Neogi, Ser-Nam Lim, and Rajiv Ramnath · 2024
Cited alongside, same era.
Towards a new research agenda for multimodal enterprise document understanding: What are we missing?
Armineh Nourbakhsh, Sameena Shah, and Carolyn Rose · 2024
Cited alongside, same era.
Anls* - A universal document processing metric for generative large language models
David Peer, Philemon Schöpf, Volckmar Nebendahl, Alexander Rietzler, and Sebastian Stabinger · 2024
Cited alongside, same era.
LMDX: language model-based document information extraction and localization
Attention, please! pixelshap reveals what vision-language models actually focus on, 2025
Roni Goldshmidt · 2025
Closest in time.
From training-free to adaptive: Empirical insights into MLLMs’ understanding of detection information, 2025
Qirui Jiao, Daoyuan Chen, Yilun Huang, Yaliang Li, and Ying Shen · 2025
Closest in time.
Smolvlm: Redefining small and efficient multimodal models
Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, and Thomas Wolf · 2025
Closest in time.
Docvlm: Make your vlm an efficient reader
Mor Shpigel Nacson, Aviad Aberdam, Roy Ganz, Elad Ben Avraham, Alona Golts, Yair Kittenplon, Shai Mazor, and Ron Litman · 2025
Closest in time.
Smoldocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Zifeng Wang, Jiaqi Mu, Hao Zhang, Chen-Yu Lee, and Nan Hua · 2024
Cited alongside, same era.
Docxplain: A novel model-agnostic explainability method for document image classification
Saifullah Saifullah, Stefan Agne, Andreas Dengel, and Sheraz Ahmed · 2024
Cited alongside, same era.
Docllm: a layout-aware generative language model for multimodal document understanding
D. Wang, N. Raman, M. Sibue, Z. Ma, P. Babkin, S. Kaur, Y. Pei, A. Nourbakhsh, and X. Liu · 2024
Cited alongside, same era.
Claude Sonnet 4 System Card: Anthropic Opus 4 Sonnet 4
Anthropic · 2025
Cited alongside, same era.
Qwen2.5-vl technical report, 2025
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin · 2025
Cited alongside, same era.
LLM-wrapper: Black-box semantic-aware adaptation of vision-language models for referring expression comprehension
Amaia Cardiel, Eloi Zablocki, Elias Ramzi, Oriane Siméoni, and Matthieu Cord · 2025
Cited alongside, same era.
Boundingdocs: a unified dataset for document question answering with spatial annotations, 2025
S. Giovannini, F. Coppini, A. Gemelli, and S. Marinai · 2025
Cited alongside, same era.
Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, Yen-Sung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Borchardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross B. Girshick, Ali Farhadi, and Aniruddha Kembhavi
Cited in the paper.
Ahmed Nassar, Andrés Marafioti, Matteo Omenetti, Maksym Lysak, Nikolaos Livathinos, Christoph Auer, Lucas Morin, Rafael Teixeira de Lima, Yusik Kim, A. Said Gurbuz, Michele Dolfi, Miquel Farré, and Peter W. J. Staar · 2025
Closest in time.
Where is this coming from? making groundedness count in the evaluation of document VQA models
Armineh Nourbakhsh, Siddharth Parekh, Pranav Shetty, Zhao Jin, Sameena Shah, and Carolyn P. Rosé · 2025
Closest in time.
Evaluating robustness of llms in question answering on multilingual noisy ocr data, 2025
Bhawna Piryani, Jamshid Mozafari, Abdelrahman Abdallah, Antoine Doucet, and Adam Jatowt · 2025
Closest in time.
Rahul Ramachandran, Ali Garjani, Roman Bachmann, Andrei Atanov, Oguzhan Fatih Kar, and Amir Zamir · 2025
Closest in time.
Bigdocs: An open dataset for training multimodal models on document and code tasks
Juan A. Rodríguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte Suresh, François Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats Leon Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, and et al · 2025
Closest in time.
DocVXQA: Context-aware visual explanations for document question answering
Mohamed Ali Souibgui, Changkyu Choi, Andrey Barsky, Kangsoo Jung, Ernest Valveny, and Dimosthenis Karatzas · 2025
Closest in time.
Michael Tschannen, Alexey A. Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier J. Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai · 2025
Closest in time.