Fetching the paper…
Reading the bibliography…
Existing benchmarks often highlight the remarkable performance achieved by state-of-the-art Multimodal Foundation Models (MFMs) in leveraging temporal context for video understanding.
Clevrer: Collision events for video representation and reasoning, 2020
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum · 1910
Earlier work this paper cites.
VIBE: video inference for human body pose and shape estimation
Muhammed Kocabas, Nikos Athanasiou, and Michael J. Black · 1912
Earlier work this paper cites.
The reviewing of object files: Object-specific integration of information
Daniel Kahneman, Anne Treisman, and Brian J Gibbs · 1992
Earlier work this paper cites.
Activitynet: A large-scale video benchmark for human activity understanding
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles · 2015
Earlier work this paper cites.
Keep it SMPL: automatic estimation of 3d human pose and shape from a single image
Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter V. Gehler, Javier Romero, and Michael J. Black · 2016
Earlier work this paper cites.
Tgif: A new dataset and benchmark on animated gif description, 2016
Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Alejandro Jaimes, and Jiebo Luo · 2016
Earlier work this paper cites.
Shuffle and learn: Unsupervised learning using temporal order verification, 2016
Ishan Misra, C. Lawrence Zitnick, and Martial Hebert · 2016
Earlier work this paper cites.
The “something something” video database for learning and evaluating visual common sense
Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fründ, Peter N. Yianilos, Moritz Mueller-Freitag, Florian Hoppe, Christian Thurau, Ingo Bax, and Roland Memisevic · 2017
Earlier work this paper cites.
Tgif-qa: Toward spatio-temporal reasoning in visual question answering, 2017
Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim · 2017
Earlier work this paper cites.
What makes a video a video: Analyzing temporal information in video understanding models and datasets
De-An Huang, Vignesh Ramanathan, Dhruv Mahajan, Lorenzo Torresani, Manohar Paluri, Li Fei-Fei, and Juan Carlos Niebles · 2018
Earlier work this paper cites.
Actor and observer: Joint modeling of first and third-person videos, 2018
Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteek Alahari · 2018
Earlier work this paper cites.
No frame left behind: Full video action recognition, 2021
Xin Liu, Silvia L. Pintea, Fatemeh Karimi Nejadasl, Olaf Booij, and Jan C. van Gemert · 2021
Earlier work this paper cites.
Scvrl: Shuffled contrastive video representation learning, 2022
Michael Dorkenwald, Fanyi Xiao, Biagio Brattoli, Joseph Tighe, and Davide Modolo · 2022
Earlier work this paper cites.
Generating diverse and natural 3d human motions from text
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng · 2022
Cited alongside, same era.
Jiachang Hao, Haifeng Sun, Pengfei Ren, Jingyu Wang, Qi Qi, and Jianxin Liao · 2022
Cited alongside, same era.
Revealing single frame bias for video-and-language learning, 2022
Jie Lei, Tamara L. Berg, and Mohit Bansal · 2022
Cited alongside, same era.
Learning to answer questions in dynamic audio-visual scenarios, 2022
Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, and Di Hu · 2022
Cited alongside, same era.
Vitatecs: A diagnostic dataset for temporal concept understanding of video-language models, 2023
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun · 2024
Closest in time.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024
Gemini · 2024
Closest in time.
Mmworld: Towards multi-discipline multi-faceted world model evaluation in videos, 2024
Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, Kevin Lin, William Yang Wang, Lijuan Wang, and Xin Eric Wang · 2024
Closest in time.
Tempcompass: Do video llms really understand videos?, 2024
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Shicheng Li, Lei Li, Shuhuai Ren, Yuanxin Liu, Yi Liu, Rundong Gao, Xu Sun, and Lu Hou · 2023
Cited alongside, same era.
Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, and Li Yuan · 2023
Cited alongside, same era.
Perception test: A diagnostic benchmark for multimodal video models, 2023
Viorica Pătrăucean, Lucas Smaira, Ankush Gupta, Adrià Recasens Continente, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Joseph Heyward, Mateusz Malinowski, Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, Antoine Miech, Alex Frechette, Hanna Klimczak, Raphael Koster, Junlin Zhang, Stephanie Winkler, Yusuf Aytar, Simon Osindero, Dima Damen, Andrew Zisserman, and João Carreira · 2023
Cited alongside, same era.
Movqa: A benchmark of versatile question-answering for long-form movie understanding, 2023
Hongjie Zhang, Yi Liu, Lu Dong, Yifei Huang, Zhen-Hua Ling, Yali Wang, Limin Wang, and Yu Qiao · 2023
Cited alongside, same era.
Auroracap: Efficient, performant video detailed captioning and a new benchmark, 2024
Wenhao Chai, Enxin Song, Yilun Du, Chenlin Meng, Vashisht Madhavan, Omer Bar-Tal, Jeng-Neng Hwang, Saining Xie, and Christopher D. Manning · 2024
Cited alongside, same era.
Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms, 2024
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing · 2024
Cited alongside, same era.
Mmbench-video: A long-form multi-shot benchmark for holistic video understanding, 2024
Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen · 2024
Cited alongside, same era.
Jiajun Fei, Dian Li, Zhidong Deng, Zekun Wang, Gang Liu, and Hui Wang · 2024
Cited alongside, same era.
Neptune: The long orbit to benchmarking long video understanding
Arsha Nagrani, Mingda Zhang, Ramin Mehran, Rachel Hornung, Nitesh Bharadwaj Gundavarapu, Nilpa Jha, Austin Myers, Xingyi Zhou, Boqing Gong, Cordelia Schmid, Mikhail Sirotenko, Yukun Zhu, and Tobias Weyand · 2024
Closest in time.
Hello gpt-4o, 2024
OpenAI · 2024
Closest in time.
Cinepile: A long video question answering dataset and benchmark, 2024
Ruchit Rawal, Khalid Saifullah, Ronen Basri, David Jacobs, Gowthami Somepalli, and Tom Goldstein · 2024
Closest in time.
Reka core, flash, and edge: A series of powerful multimodal language models, 2024
Reka · 2024
Closest in time.
Zhi Rui Tam, Cheng-Kuang Wu, Yi-Lin Tsai, Chieh-Yen Lin, Hung yi Lee, and Yun-Nung Chen · 2024
Closest in time.
Phi-3 technical report: A highly capable language model locally on your phone, 2024
Team · 2024
Closest in time.
Llava-next: A strong zero-shot video understanding model, April 2024a
Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li · 2024
Closest in time.
Mlvu: A comprehensive benchmark for multi-task long video understanding, 2024
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu · 2024
Closest in time.