Fetching the paper…
Reading the bibliography…
Vision-Language Models (VLMs) have achieved notable success in multimodal tasks but face practical limitations due to the quadratic complexity of decoder attention mechanisms and autoregressive generation.
“Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,”
Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik, · 2015
Earlier work this paper cites.
“Dynamicvit: Efficient vision transformers with dynamic token sparsification,”
Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh, · 2021
Earlier work this paper cites.
“Llm.int8(): 8-bit matrix multiplication for transformers at scale,”
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer, · 2022
Earlier work this paper cites.
“GPTQ: accurate post-training quantization for generative pre-trained transformers,”
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh, · 2022
Earlier work this paper cites.
“Evit: Expediting vision transformers via token reorganizations,”
Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie, · 2022
Earlier work this paper cites.
“Learned token pruning for transformers,”
Sehoon Kim, Sheng Shen, David Thorsley, Amir Gholami, Woosuk Kwon, Joseph Hassoun, and Kurt Keutzer, · 2022
Earlier work this paper cites.
“Groupvit: Semantic segmentation emerges from text supervision,”
Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas M. Breuel, Jan Kautz, and Xiaolong Wang, · 2022
Earlier work this paper cites.
“Qwen-vl: A frontier large vision-language model with versatile abilities,”
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou, · 2023
Earlier work this paper cites.
“Video-llava: Learning united visual representation by alignment before projection,”
Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan, · 2023
Earlier work this paper cites.
“Llava-plus: Learning to use tools for creating multimodal agents,”
Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, et al., · 2023
Earlier work this paper cites.
“Video-llama: An instruction-tuned audio-visual language model for video understanding,”
Hang Zhang, Xin Li, and Lidong Bing, · 2023
Cited alongside, same era.
“Mobilevlm : A fast, strong and open vision language assistant for mobile devices,”
Xiangxiang Chu, Limeng Qiao, Xinyang Lin, Shuang Xu, Yang Yang, Yiming Hu, Fei Wei, Xinyu Zhang, Bo Zhang, Xiaolin Wei, and Chunhua Shen, · 2023
Cited alongside, same era.
“Llm-pruner: On the structural pruning of large language models,”
Xinyin Ma, Gongfan Fang, and Xinchao Wang, · 2023
Cited alongside, same era.
“Diffrate : Differentiable compression rate for efficient vision transformers,”
Mengzhao Chen, Wenqi Shao, Peng Xu, Mingbao Lin, Kaipeng Zhang, Fei Chao, Rongrong Ji, Yu Qiao, and Ping Luo, · 2023
Cited alongside, same era.
“Token merging: Your vit but faster,”
Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman, · 2023
Cited alongside, same era.
“Vl-mamba: Exploring state space models for multimodal learning,”
Yanyuan Qiao, Zheng Yu, Longteng Guo, Sihan Chen, Zijia Zhao, Mingzhen Sun, Qi Wu, and Jing Liu, · 2024
Later among the works it cites.
“Llava-phi: Efficient multi-modal assistant with small language model,”
Yichen Zhu, Minjie Zhu, Ning Liu, Zhicai Ou, Xiaofeng Mou, and Jian Tang, · 2024
Later among the works it cites.
“Self-distillation bridges distribution gap in language model fine-tuning,”
Zhaorui Yang, Qian Liu, Tianyu Pang, Han Wang, Haozhe Feng, Minfeng Zhu, and Wei Chen, · 2024
Later among the works it cites.
“Llava-prumerge: Adaptive token reduction for efficient large multimodal models,”
Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan, · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang, · 2024
Cited alongside, same era.
“Boosting multimodal large language models with visual tokens withdrawal for rapid inference,”
Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji, · 2024
Cited alongside, same era.
“A survey on efficient inference for large language models,”
Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang, · 2024
Cited alongside, same era.
“Beyond llava-hd: Diving into high-resolution large multimodal models,”
Yi-Fan Zhang, Qingsong Wen, Chaoyou Fu, Xue Wang, Zhang Zhang, Liang Wang, and Rong Jin, · 2024
Cited alongside, same era.
“Moe-llava: Mixture of experts for large vision-language models,”
Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Junwu Zhang, Munan Ning, and Li Yuan, · 2024
Cited alongside, same era.
Jianjian Cao, Peng Ye, Shengze Li, Chong Yu, Yansong Tang, Jiwen Lu, and Tao Chen, · 2024
Later among the works it cites.
“Tokenpacker: Efficient visual projector for multimodal llm,”
Wentong Li, Yuqian Yuan, Jian Liu, Dongqi Tang, Song Wang, Jianke Zhu, and Lei Zhang, · 2024
Later among the works it cites.
“Sparsevlm: Visual token sparsification for efficient vision-language model inference,”
Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al., · 2024
Later among the works it cites.
“Visionzip: Longer is better but not necessary in vision language models,”
Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia, · 2024
Later among the works it cites.
“From redundancy to relevance: Enhancing explainability in multimodal large language models,”
Xiaofeng Zhang, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, and Jieping Ye, · 2024
Later among the works it cites.
“Semantic role labeling from Chinese speech via end-to-end learning,”
Huiyao Chen, Xinxin Li, Meishan Zhang, and Min Zhang, · 2024
Later among the works it cites.