Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation

Sentence-bert: Sentence embeddings using siamese bert-networks

Original

N Reimers. 2019 · 1908

Earlier work this paper cites.

A diagram is worth a dozen images

Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016 · 2016

Earlier work this paper cites.

Figureseer: Parsing result-figures in research papers

Noah Siegel, Zachary Horvitz, Roie Levin, Santosh Divvala, and Ali Farhadi. 2016 · 2016

Earlier work this paper cites.

Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering

Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017 · 2017

Earlier work this paper cites.

Clevr: A diagnostic dataset for compositional language and elementary visual reasoning

Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017 · 2017

Earlier work this paper cites.

Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks?

Matthew Johnson-Roberson, Charles Barto, Rounak Mehta, Sharath Nittur Sridhar, Karl Rosaen, and Ram Vasudevan. 2017 · 2017

Earlier work this paper cites.

FigureQA: An annotated figure dataset for visual reasoning

Original

Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2017 · 2017

Earlier work this paper cites.

Vizwiz grand challenge: Answering visual questions from blind people

Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. 2018 · 2018

Earlier work this paper cites.

DVQA: Understanding data visualizations via question answering

Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018 · 2018

Earlier work this paper cites.

Know what you don’t know: Unanswerable questions for SQuAD

Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018 · 2018

Earlier work this paper cites.

TallyQA: Answering complex counting questions

Manoj Acharya, Kushal Kafle, and Christopher Kanan. 2019 · 2019

Earlier work this paper cites.

Scene text visual question answering

Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marçal Rusinol, Ernest Valveny, CV Jawahar, and Dimosthenis Karatzas. 2019 · 2019

Earlier work this paper cites.

Gqa: A new dataset for real-world visual reasoning and compositional question answering

Drew A Hudson and Christopher D Manning. 2019 · 2019

Earlier work this paper cites.

OK-VQA: A visual question answering benchmark requiring external knowledge

Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019 · 2019

Earlier work this paper cites.

Ocr-vqa: Visual question answering by reading text in images

Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. 2019 · 2019

Earlier work this paper cites.

Towards VQA models that can read

Amanpreet Singh, Vivek Natarjan, Meet Shah, Yu Jiang, Xinlei Chen, Devi Parikh, and Marcus Rohrbach. 2019 · 2019

Earlier work this paper cites.

PlotQA: Reasoning over scientific plots

Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. 2020 · 2020

Earlier work this paper cites.

Exploring the limits of transfer learning with a unified text-to-text transformer

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020 · 2020

Earlier work this paper cites.

Stl-cqa: Structure-based transformers with localization and encoding for chart question answering

Hrituraj Singh and Sumit Shekhar. 2020 · 2020

Earlier work this paper cites.

DocVQA: A dataset for VQA on document images

Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021 · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021 · 2021

Earlier work this paper cites.

Multimodal few-shot learning with frozen language models

Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021 · 2021

Earlier work this paper cites.

Flamingo: a visual language model for few-shot learning

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022 · 2022

Cited alongside, same era.

Simvqa: Exploring simulated environments for visual question answering

Paola Cascante-Bonilla, Hui Wu, Letao Wang, Rogerio S Feris, and Vicente Ordonez. 2022 · 2022

Cited alongside, same era.

Learn to explain: Multimodal reasoning via thought chains for science question answering

Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022 · 2022

Cited alongside, same era.

ChartQA: A benchmark for question answering about charts with visual and logical reasoning

Ahmed Masry, Do Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022 · 2022

Cited alongside, same era.

InfographicVQA

Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. 2022 · 2022

Cited alongside, same era.

Seeclick: Harnessing gui grounding for advanced visual gui agents

Original

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024 · 2024

Later among the works it cites.

Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

Original

Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. 2024 · 2024

Later among the works it cites.

Scaling synthetic data creation with 1,000,000,000 personas

Original

Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. 2024 · 2024

Later among the works it cites.

Navigating the digital world as humans do: Universal visual grounding for gui agents

Original

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. 2024 · 2024

Later among the works it cites.