Fetching the paper…
Reading the bibliography…
The deployment of Large Multimodal Models (LMMs) within AntGroup has significantly advanced multimodal tasks in payment, security, and advertising, notably enhancing advertisement audition tasks in Alipay.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
Structured compression by weight encryption for unstructured pruning and quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1909–1918
Se Jung Kwon, Dongsoo Lee, Byeongwook Kim, Parichay Kapoor, Baeseong Park, and Gu-Yeon Wei. 2020 · 1918
Earlier work this paper cites.
Notes on kullback-leibler divergence and likelihood
Jonathon Shlens. 2014 · 2014
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015
Earlier work this paper cites.
Structured pruning of deep convolutional neural networks
Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. 2017 · 2017
Earlier work this paper cites.
Reducing transformer depth on demand with structured dropout
Angela Fan, Edouard Grave, and Armand Joulin. 2019 · 2019
Earlier work this paper cites.
Structured pruning of a bert-based question answering model
JS McCarley, Rishav Chakravarti, and Avirup Sil. 2019 · 2019
Earlier work this paper cites.
Importance estimation for neural network pruning. In Proceedings of the CVPR . 11264–11272
Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz. 2019 · 2019
Earlier work this paper cites.
Patient Knowledge Distillation for BERT Model Compression. In (EMNLP-IJCNLP) . Association for Computational Linguistics
Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. 2019 · 2019
Earlier work this paper cites.
Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019 · 2019
Earlier work this paper cites.
Eigendamage: Structured pruning in the kronecker-factored eigenbasis. In International conference on machine learning . PMLR, 6566–6575
Chaoqi Wang, Roger Grosse, Sanja Fidler, and Guodong Zhang. 2019 · 2019
Earlier work this paper cites.
Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning
Mitchell A Gordon, Kevin Duh, and Nicholas Andrews. 2020 · 2020
Earlier work this paper cites.
Dynabert: Dynamic bert with adaptive width and depth
Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. 2020 · 2020
Earlier work this paper cites.
TinyBERT: Distilling BERT for Natural Language Understanding. In EMNLP 2020 . 4163–4174
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020 · 2020
Earlier work this paper cites.
Layer-adaptive sparsity for the magnitude-based pruning
Jaeho Lee, Sejun Park, Sangwoo Mo, Sungsoo Ahn, and Jinwoo Shin. 2020 · 2020
Earlier work this paper cites.
Bingbing Li, Zhenglun Kong, Tianyun Zhang, Ji Li, Zhengang Li, Hang Liu, and Caiwen Ding. 2020 · 2020
Earlier work this paper cites.
Knowledge distillation and data augmentation for NLP light pre-trained models. In Journal of Physics: Conference Series , Vol. 1651. IOP Publishing, 012043
Hanwen Luo, Yudong Li, Xiaodong Wang, and Yuqing Zhang. 2020 · 2020
Earlier work this paper cites.
Q-bert: Hessian based ultra low precision quantization of bert. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 34. 8815–8821
Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. 2020 · 2020
Cited alongside, same era.
Distilling knowledge from well-informed soft labels for neural relation extraction. In Proceedings of the AAAI , Vol. 34. 9620–9627
Zhenyu Zhang, Xiaobo Shu, Bowen Yu, Tingwen Liu, Jiapeng Zhao, Quangang Li, and Li Guo. 2020 · 2020
Cited alongside, same era.
I-bert: Integer-only bert quantization. In International conference on machine learning . PMLR, 5506–5518
Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2021 · 2021
Cited alongside, same era.
EBERT: Efficient BERT inference with dynamic structured pruning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . 4814–4823
Zejian Liu, Fanrong Li, Gang Li, and Jian Cheng. 2021 · 2021
Cited alongside, same era.
Homodistil: Homotopic task-agnostic distillation of pre-trained transformers
Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bin Yin, and Tuo Zhao. 2023 · 2023
Closest in time.
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023 · 2023
Closest in time.
LLM-Pruner: On the Structural Pruning of Large Language Models. In Advances in Neural Information Processing Systems
Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023 · 2023
Closest in time.
Distilling reasoning capabilities into smaller language models. In Findings of the Association for Computational Linguistics: ACL 2023 . 7059–7073
Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. 2023 · 2023
Closest in time.
A Simple and Effective Pruning Approach for Large Language Models
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Cited alongside, same era.
Prune once for all: Sparse pre-trained language models
Ofir Zafrir, Ariel Larey, Guy Boudoukh, Haihao Shen, and Moshe Wasserblat. 2021 · 2021
Cited alongside, same era.
Llm. int8 (): 8-bit matrix multiplication for transformers at scale
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022 · 2022
Cited alongside, same era.
OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation
Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, and Jiebo Luo. 2023 · 2023
Cited alongside, same era.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al · 2023
Cited alongside, same era.
SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
Luciano Del Corro, Allie Del Giorno, Sahaj Agarwal, Bin Yu, Ahmed Awadallah, and Subhabrata Mukherjee. 2023 · 2023
Cited alongside, same era.
The Use of ChatGPT to Assist in Diagnosing Glaucoma Based on Clinical Case Reports
Mohammad Delsoz, Hina Raja, Yeganeh Madadi, Anthony A Tang, Barbara M Wirostko, Malik Y Kahook, and Siamak Yousefi. 2023 · 2023
Cited alongside, same era.
DepGraph: Towards Any Structural Pruning
Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang. 2023 · 2023
Cited alongside, same era.
Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023 · 2023
Closest in time.
Prune and Tune: Improving Efficient Pruning Techniques for Massive Language Models
Aaquib Syed, Phillip Huang Guo, and Vijaykaarti Sundarapandiyan. 2023 · 2023
Closest in time.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023
Closest in time.
Smoothquant: Accurate and efficient post-training quantization for large language models. In ICML . PMLR, 38087–38099
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023 · 2023
Closest in time.
A survey on model compression and acceleration for pretrained language models. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 37. 10566–10575
Canwen Xu and Julian McAuley. 2023 · 2023
Closest in time.
Baichuan 2: Open large-scale language models
Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, et al · 2023
Closest in time.
mplug-owl: Modularization empowers large language models with multimodality
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al · 2023
Closest in time.
Large Multi-modal Encoders for Recommendation
Zixuan Yi, Zijun Long, Iadh Ounis, Craig Macdonald, and Richard Mccreadie. 2023 · 2023
Closest in time.
CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data. In Proceedings of the CVPR . 15244–15253
Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chaoqiang Ye, Qingqiu Huang, Dit-Yan Yeung, Zhen Yang, Xiaodan Liang, and Hang Xu. 2023 · 2023
Closest in time.
On the Opportunities of Green Computing: A Survey
You Zhou, Xiujing Lin, Xiang Zhang, Maolin Wang, Gangwei Jiang, Huakang Lu, Yupeng Wu, Kai Zhang, Zhe Yang, Kehang Wang, et al · 2023
Closest in time.
Minigpt-4: Enhancing vision-language understanding with advanced large language models
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023a · 2023
Closest in time.
A survey on model compression for large language models
Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2023b · 2023
Closest in time.