Fetching the paper…
Reading the bibliography…
We propose Ming-Omni, a unified multimodal model capable of processing images, text, audio, and video, while demonstrating strong proficiency in both speech and image generation.
Covost: A diverse multilingual speech-to-text translation corpus
Changhan Wang, Juan Pino, Anne Wu, and Jiatao Gu · 2002
Earlier work this paper cites.
Covost 2: A massively multilingual speech-to-text translation corpus
Changhan Wang, Anne Wu, and Juan Miguel Pino · 2007
Earlier work this paper cites.
Ted-lium: an automatic speech recognition dedicated corpus
Anthony Rousseau, Paul Deléglise, and Yannick Esteve · 2012
Earlier work this paper cites.
ReferItGame: Referring to objects in photographs of natural scenes
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg · 2014
Earlier work this paper cites.
Solving geometry problems: Combining text and diagram interpretation
Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm · 2015
Earlier work this paper cites.
Generation and comprehension of unambiguous object descriptions
Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan Yuille, and Kevin Murphy · 2016
Earlier work this paper cites.
AISHELL-1: an open-source mandarin speech corpus and a speech recognition baseline
Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng · 2017
Earlier work this paper cites.
Image-to-markup generation with coarse-to-fine attention, 2017
Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M. Rush · 2017
Earlier work this paper cites.
Video question answering with spatio-temporal reasoning
Yunseok Jang, Yale Song, Chris Dongjoo Kim, Youngjae Yu, Youngjin Kim, and Gunhee Kim · 2019
Earlier work this paper cites.
Towards vqa models that can read
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach · 2019
Earlier work this paper cites.
Beyond the nav-graph: Vision and language navigation in continuous environments
Jacob Krantz, Erik Wijmans, Arjun Majundar, Dhruv Batra, and Stefan Lee · 2020
Earlier work this paper cites.
Mls: A large-scale multilingual dataset for speech research
Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert · 2020
Earlier work this paper cites.
Aishell-3: A multi-speaker mandarin tts corpus and the baselines
Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, and Ming Li · 2020
Earlier work this paper cites.
Clevrer: Collision events for video representation and reasoning
Kexin Yi*, Chuang Gan*, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum · 2020
Earlier work this paper cites.
Overview of the vqa-med task at imageclef 2021: Visual question answering and generation in the medical domain
Asma Ben Abacha, Mourad Sarrouti, Dina Demner-Fushman, Sadid A. Hasan, and Henning Müller · 2021
Earlier work this paper cites.
Gigaspeech: An evolving, multi-domain ASR corpus with 10, 000 hours of transcribed audio
Guoguo Chen, Shuzhou Chai, Guan-Bo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Zhao You, and Zhiyong Yan · 2021
Earlier work this paper cites.
Yihui Fu, Luyao Cheng, Shubo Lv, Yukai Jv, Yuxiang Kong, Zhuo Chen, Yanxin Hu, Lei Xie, Jian Wu, Hui Bu, et al · 2021
Earlier work this paper cites.
The people’s speech: A large-scale diverse english speech recognition dataset for commercial usage
Daniel Galvez, Greg Diamos, Juan Ciro, Juan Felipe Cerón, Keith Achorn, Anjali Gopi, David Kanter, Maximilian Lam, Mark Mazumder, and Vijay Janapa Reddi · 2021
Earlier work this paper cites.
Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering, 2021
Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu · 2021
Earlier work this paper cites.
Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning, 2021
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu · 2021
Earlier work this paper cites.
Docvqa: A dataset for vqa on document images
Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar · 2021
Earlier work this paper cites.
Spgispeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition
Patrick K. O’Neill, Vitaly Lavrukhin, Somshubra Majumdar, Vahid Noroozi, Yuekai Zhang, Oleksii Kuchaiev, Jagadeesh Balam, Yuliya Dovzhenko, Keenan Freyberg, Michael D. Shulman, Boris Ginsburg, Shinji Watanabe, and Georg Kucsko · 2021
Earlier work this paper cites.
Kespeech: An open source speech dataset of mandarin and its eight subdialects
Zhiyuan Tang, Dong Wang, Yanguang Xu, Jianwei Sun, Xiaoning Lei, Shuaijiang Zhao, Cheng Wen, Xingjun Tan, Chuandong Xie, Shuran Zhou, et al · 2021
Earlier work this paper cites.
Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux · 2021
Earlier work this paper cites.
Librispeech transducer model with internal language model prior correction
Albert Zeyer, André Merboldt, Wilfried Michel, Ralf Schlüter, and Hermann Ney · 2021
Earlier work this paper cites.
Coyo-700m: Image-text pair dataset
Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim · 2022
Earlier work this paper cites.
An augmented benchmark dataset for geometric question answering through dual parallel text encoding
Jie Cao and Jing Xiao · 2022
Earlier work this paper cites.
Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression, 2022
Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang · 2022
Earlier work this paper cites.
A large-scale database for chemical structure recognition and preliminary evaluation
Longfei Ding, Mengbiao Zhao, Fei Yin, Shuiling Zeng, and Cheng-Lin Liu · 2022
Cited alongside, same era.
Wukong: A 100 million large-scale chinese cross-modal pre-training benchmark
Jiaxi Gu, Xiaojun Meng, Guansong Lu, Lu Hou, Niu Minzhe, Xiaodan Liang, Lewei Yao, Runhui Huang, Wei Zhang, Xin Jiang, et al · 2022
Cited alongside, same era.
Egotaskqa: Understanding human tasks in egocentric videos, 2022
Baoxiong Jia, Ting Lei, Song-Chun Zhu, and Siyuan Huang · 2022
Cited alongside, same era.
ChartQA: A benchmark for question answering about charts with visual and logical reasoning
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque · 2022
Cited alongside, same era.
Laion-5b: An open large-scale dataset for training next generation image-text models
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al · 2022
Document image machine translation with dynamic multi-pre-trained models assembling
Yupu Liang, Yaping Zhang, Cong Ma, Zhiyang Zhang, Yang Zhao, Lu Xiang, Chengqing Zong, and Yu Zhou · 2024
Later among the works it cites.
Tempcompass: Do video llms really understand videos?, 2024
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou · 2024
Later among the works it cites.
Videogpt+: Integrating image and video encoders for enhanced video understanding
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan · 2024
Later among the works it cites.
Openvid-1m: A large-scale high-quality dataset for text-to-video generation
Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai · 2024
Later among the works it cites.
Omnidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Open source magicdata-ramc: A rich annotated mandarin conversational(ramc) speech dataset
Zehui Yang, Yifan Chen, Lei Luo, Runyan Yang, Lingxuan Ye, Gaofeng Cheng, Ji Xu, Yaohui Jin, Qingqing Zhang, Pengyuan Zhang, Lei Xie, and Yonghong Yan · 2022
Cited alongside, same era.
M2met: The icassp 2022 multi-channel multi-party meeting transcription challenge
Fan Yu, Shiliang Zhang, Yihui Fu, Lei Xie, Siqi Zheng, Zhihao Du, Weilong Huang, Pengcheng Guo, Zhijie Yan, Bin Ma, Xin Xu, and Hui Bu · 2022
Cited alongside, same era.
Instructpix2pix: Learning to follow image editing instructions
Tim Brooks, Aleksander Holynski, and Alexei A Efros · 2023
Cited alongside, same era.
Can pre-trained vision and language models answer visual information-seeking questions?
Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, and Ming-Wei Chang · 2023
Cited alongside, same era.
Opencompass: A universal evaluation platform for foundation models
OpenCompass Contributors · 2023
Cited alongside, same era.
Funasr: A fundamental end-to-end speech recognition toolkit, 2023
Zhifu Gao, Zerui Li, Jiaming Wang, Haoneng Luo, Xian Shi, Mengzhe Chen, Yabin Li, Lingyun Zuo, Zhihao Du, Zhangyu Xiao, and Shiliang Zhang · 2023
Cited alongside, same era.
Geomverse: A systematic evaluation of large models for geometric reasoning, 2023
Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut · 2023
Cited alongside, same era.
Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, and Conghui He · 2024
Later among the works it cites.
Tokenflow: Unified image tokenizer for multimodal understanding and generation
Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu · 2024
Later among the works it cites.
Cinepile: A long video question answering dataset and benchmark, 2024
Ruchit Rawal, Khalid Saifullah, Miquel Farré, Ronen Basri, David Jacobs, Gowthami Somepalli, and Tom Goldstein · 2024
Later among the works it cites.
How to train data-efficient llms
Noveen Sachdeva, Benjamin Coleman, Wang-Cheng Kang, Jianmo Ni, Lichan Hong, Ed Huai hsin Chi, James Caverlee, Julian J. McAuley, and Derek Zhiyuan Cheng · 2024
Later among the works it cites.
Metamorph: Multimodal understanding and generation via instruction tuning
Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu · 2024
Later among the works it cites.
Slidespeech: A large scale slide-enriched audio-visual corpus
Haoxu Wang, Fan Yu, Xian Shi, Yuezhang Wang, Shiliang Zhang, and Ming Li · 2024
Later among the works it cites.
Funqa: Towards surprising video comprehension, 2024
Binzhu Xie, Sicheng Zhang, Zitang Zhou, Bo Li, Yuanhan Zhang, Jack Hessel, Jingkang Yang, and Ziwei Liu · 2024
Later among the works it cites.
Vript: A video is worth thousands of words
Dongjie Yang, Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao · 2024
Later among the works it cites.
Ultraedit: Instruction-based fine-grained image editing at scale
Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang · 2024
Later among the works it cites.
Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, et al · 2025
Closest in time.
M2-omni: Advancing omni-mllm for comprehensive modality support with competitive performance
Qingpei Guo, Kaiyou Song, Zipeng Feng, Ziping Ma, Qinglong Zhang, Sirui Gao, Xuzheng Yu, Yunxiao Sun, Tai-Wei Chang, Jingdong Chen, et al · 2025
Closest in time.
Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos
Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu · 2025
Closest in time.
Step-audio: Unified understanding and generation in intelligent speech interaction
Ailin Huang, Boyong Wu, Bruce Wang, Chao Yan, Chen Hu, Chengli Feng, Fei Tian, Feiyu Shen, Jingbei Li, Mingrui Chen, et al · 2025
Closest in time.
Ming-lite-uni: Advancements in unified architecture for natural multimodal interaction, 2025
InclusionAI, Biao Gong, Cheng Zou, Dandan Zheng, Hu Yu, Jingdong Chen, Jianxin Sun, Junbo Zhao, Jun Zhou, Kaixiang Ji, Lixiang Ru, Libin Wang, Qingpei Guo, Rui Liu, Weilong Chai, Xinyu Xiao, and Ziyuan Huang · 2025
Closest in time.
KimiTeam, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al · 2025
Closest in time.
Chemvlm: Exploring the power of multimodal large language models in chemistry area, 2025
Junxian Li, Di Zhang, Xunzhi Wang, Zeying Hao, Jingdi Lei, Qian Tan, Cai Zhou, Wei Liu, Yaotian Yang, Xinrui Xiong, Weiyun Wang, Zhe Chen, Wenhai Wang, Wei Li, Shufei Zhang, Mao Su, Wanli Ouyang, Yuqiang Li, and Dongzhan Zhou · 2025
Closest in time.
Every flop counts: Scaling a 300b mixture-of-experts ling llm without premium gpus
LingTeam, Binwei Zeng, Chao Huang, Chao Zhang, Changxin Tian, Cong Chen, Dingnan Jin, Feng Yu, Feng Zhu, Feng Yuan, et al · 2025
Closest in time.
Neptune: The long orbit to benchmarking long video understanding, 2025
Arsha Nagrani, Mingda Zhang, Ramin Mehran, Rachel Hornung, Nitesh Bharadwaj Gundavarapu, Nilpa Jha, Austin Myers, Xingyi Zhou, Boqing Gong, Cordelia Schmid, Mikhail Sirotenko, Yukun Zhu, and Tobias Weyand · 2025
Closest in time.
Introducing 4o image generation
OpenAI · 2025
Closest in time.
Transfer between modalities with metaqueries
Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al · 2025
Closest in time.
Jiatong Shi, Chunlei Zhang, Jinchuan Tian, Junrui Ni, Hao Zhang, Shinji Watanabe, and Dong Yu · 2025
Closest in time.
Qwen2. 5-omni technical report
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, et al · 2025
Closest in time.
R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al · 2025
Closest in time.