Fetching the paper…
Reading the bibliography…
Current foundation models exhibit impressive capabilities when prompted either with text only or with both image and text inputs.
The picture superiority effect in recognition memory: A developmental study using the response signal procedure
Margaret Anne Defeyter, Riccardo Russo, and Pamela Louise McPartlin · 2009
Earlier work this paper cites.
An asymmetrical relationship between verbal and visual thinking: Converging evidence from behavior and fmri
Elinor Amit, Caitlyn Hoeflin, Nada Hamzah, and Evelina Fedorenko · 2017
Earlier work this paper cites.
Shifting the baseline: Single modality performance on visual navigation & qa
Jesse Thomason, Daniel Gordon, and Yonatan Bisk · 2018
Earlier work this paper cites.
Zoom in: An introduction to circuits
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter · 2020
Earlier work this paper cites.
Reframing instructional prompts to gptk’s language
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi · 2021
Earlier work this paper cites.
Gpt3mix: Leveraging large-scale language models for text augmentation
Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyeong Park · 2021
Earlier work this paper cites.
Pali: A jointly-scaled multilingual language-image model
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al · 2022
Earlier work this paper cites.
What can transformers learn in-context? a case study of simple function classes
Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant · 2022
Earlier work this paper cites.
Demystifying prompts in language models via perplexity estimation
Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer · 2022
Earlier work this paper cites.
Xiaochuang Han and Yulia Tsvetkov · 2022
Earlier work this paper cites.
Countr: Transformer-based generalised visual counting
Chang Liu, Yujie Zhong, Andrew Zisserman, and Weidi Xie · 2022
Earlier work this paper cites.
Learn to explain: Multimodal reasoning via thought chains for science question answering
Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan · 2022
Earlier work this paper cites.
Impact of pretraining term frequencies on few-shot reasoning
Yasaman Razeghi, Robert L Logan IV, Matt Gardner, and Sameer Singh · 2022
Earlier work this paper cites.
Transformers learn in-context by gradient descent
Johannes von Oswald, Eyvind Niklasson, E. Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov · 2022
Earlier work this paper cites.
Palm 2 technical report, 2023
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu · 2023
Earlier work this paper cites.
Introducing our multimodal models, 2023
Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sağnak Taşırlar · 2023
Earlier work this paper cites.
Simplicity bias in transformers and their ability to learn sparse Boolean functions
Satwik Bhattamishra, Arkil Patel, Varun Kanade, and Phil Blunsom · 2023
Cited alongside, same era.
Data curation alone can stabilize in-context learning
Ting-Yun Chang and Robin Jia · 2023
Cited alongside, same era.
Pali-x: On scaling up a multilingual vision and language model
Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, et al · 2023
Cited alongside, same era.
Liying Cheng, Xingxuan Li, and Lidong Bing · 2023
Cited alongside, same era.
Transformers learn higher-order optimization methods for in-context learning: A study with linear models, 2023
Deqing Fu, Tian-Qi Chen, Robin Jia, and Vatsal Sharan · 2023
Cited alongside, same era.
Scaling autoregressive multi-modal models: Pretraining and instruction tuning
Lili Yu, Bowen Shi, Ramakanth Pasunuru, Benjamin Muller, Olga Golovneva, Tianlu Wang, Arun Babu, Binh Tang, Brian Karrer, Shelly Sheynin, et al · 2023
Later among the works it cites.
MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al · 2023
Later among the works it cites.
The claude 3 model family: Opus, sonnet, haiku
Anthropic · 2024
Closest in time.
Towards understanding the word sensitivity of attention layers: A study via random features, 2024
Simone Bombari and Marco Mondelli · 2024
Closest in time.
Nphardeval4v: A dynamic reasoning benchmark of multimodal large language models
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Gemini: A family of highly capable multimodal models, 2023
Gemini Team Google · 2023
Cited alongside, same era.
Instruction tuning with lexicons for zero-shot style classification
Ruohao Guo, Wei Xu, and Alan Ritter · 2023
Cited alongside, same era.
How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
Michael Hanna, Ollie Liu, and Alexandre Variengien · 2023
Cited alongside, same era.
Neftune: Noisy embeddings improve instruction finetuning
Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, et al · 2023
Cited alongside, same era.
Approximating cky with transformers
Ghazal Khalighinejad, Ollie Liu, and Sam Wiseman · 2023
Cited alongside, same era.
Improved instruction ordering in recipe-grounded conversation
Duong Minh Le, Ruohao Guo, Wei Xu, and Alan Ritter · 2023
Cited alongside, same era.
Teaching arithmetic to small transformers
Nayoung Lee, Kartik K. Sreenivasan, Jason D. Lee, Kangwook Lee, and Dimitris Papailiopoulos · 2023
Cited alongside, same era.
Lizhou Fan, Wenyue Hua, Xiang Li, Kaijie Zhu, Mingyu Jin, Lingyao Li, Haoyang Ling, Jinkui Chi, Jindong Wang, Xin Ma, et al · 2024
Closest in time.
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al · 2024
Closest in time.
Interpretable diffusion via information decomposition
Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, and Greg Ver Steeg · 2024
Closest in time.
Quantifying & modeling multimodal interactions: An information decomposition framework
Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy Auerbach, Faisal Mahmood, et al · 2024
Closest in time.
Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee · 2024
Closest in time.
MathVista: Evaluating mathematical reasoning of foundation models in visual contexts, 2024
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao · 2024
Closest in time.
Mm1: Methods, analysis & insights from multimodal llm pre-training
Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, et al · 2024
Closest in time.
Reka flash: An efficient and capable multimodal language model
Reka · 2024
Closest in time.
Chameleon: Mixed-modal early-fusion foundation models
Chameleon Team · 2024
Closest in time.
Simplicity bias of transformers to learn low sensitivity functions, 2024
Bhavya Vasudeva, Deqing Fu, Tianyi Zhou, Elliott Kau, Youqi Huang, and Vatsal Sharan · 2024
Closest in time.
Pre-trained large language models use fourier features to compute addition, 2024
Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia · 2024
Closest in time.