mDPO: Conditional Preference Optimization for Multimodal Large Language Models

Rank analysis of incomplete block designs: I. the method of paired comparisons

Ralph Allan Bradley and Milton E Terry. 1952 · 1952

Earlier work this paper cites.

Improved baselines with momentum contrastive learning

Original

Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020 · 2003

Earlier work this paper cites.

Deep reinforcement learning from human preferences

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017 · 2017

Earlier work this paper cites.

Object hallucination in image captioning

Original

Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018 · 2018

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021 · 2021

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 · 2023

Earlier work this paper cites.

Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023 · 2023

Earlier work this paper cites.

Phi-2: The surprising power of small language models

Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio César Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. 2023 · 2023

Earlier work this paper cites.

Silkie: Preference distillation for large visual language models

Original

Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, and Lingpeng Kong. 2023 · 2023

Earlier work this paper cites.

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023 · 2023

Earlier work this paper cites.

Aligning large multimodal models with factually augmented rlhf

Original

Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. 2023 · 2023

Earlier work this paper cites.

An llm-free multi-dimensional benchmark for mllms hallucination evaluation

Original

Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. 2023 · 2023

Earlier work this paper cites.

Some things are more cringe than others: Preference optimization with the pairwise cringe loss

Original

Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, and Jason Weston. 2023 · 2023

Cited alongside, same era.

Sigmoid loss for language image pre-training

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023 · 2023

Cited alongside, same era.

Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization

Original

Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. 2023 · 2023

Cited alongside, same era.

A general theoretical paradigm to understand learning from human preferences

Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024 · 2024

Cited alongside, same era.

Training small multimodal models to bridge biomedical competency gap: A case study in radiology imaging

Original

Mitigating object hallucinations in large vision-language models through visual contrastive decoding

Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024 · 2024

Closest in time.

Llava-onevision: Easy visual task transfer

Original

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024 · 2024

Closest in time.

Simpo: Simple preference optimization with a reference-free reward

Original

Yu Meng, Mengzhou Xia, and Danqi Chen. 2024 · 2024

Closest in time.

Disentangling length from quality in direct preference optimization

Original

Ryan Park, Rafael Rafailov, Stefano Ermon, and Chelsea Finn. 2024 · 2024

Closest in time.

Strengthening multimodal large language model with bootstrapped preference optimization