Fetching the paper…

Reading the bibliography…

2024

VLRM: Vision-Language Models act as Reward Models for Image Captioning

Dzabraev, Maksim, Kunitsyn, Alexander, Ivaniuta, Andrei

Original
PDF
Post on X

Understand

In this work, we present an unsupervised method for enhancing an image captioning model (in our case, BLIP2) using reinforcement learning and vision-language models like CLIP and BLIP2-ITM as reward models.

The RL-tuned model is able to generate longer and more comprehensive descriptions.
Our model reaches impressive 0.90 R@1 CLIP Recall score on MS-COCO Carpathy Test Split.
Weights are available at https://huggingface.co/sashakunitsyn/vlrm-blip2-opt-2.7b.

VLRM: Vision-Language Models act as Reward Models for Image Captioning · Around

Built on

“Microsoft COCO: Common Objects in Context”, 2015
Original
Tsung-Yi Lin et al · 2015
Earlier work this paper cites.
“Asynchronous Methods for Deep Reinforcement Learning”, 2016
Original
Volodymyr Mnih et al · 2016
Earlier work this paper cites.
“Adam: A Method for Stochastic Optimization”, 2017
Original
Diederik. Kingma and Jimmy Ba · 2017
Earlier work this paper cites.
“Proximal Policy Optimization Algorithms”, 2017
Original
John Schulman et al · 2017
Earlier work this paper cites.
“Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts”, 2021
Original
Soravit Changpinyo, Piyush Sharma, Nan Ding and Radu Soricut · 2021
Earlier work this paper cites.
“OpenCLIP” If you use this software, please cite it as below
Gabriel Ilharco et al · 2021
Earlier work this paper cites.
“Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision”, 2021
Original
Chao Jia et al · 2021
Earlier work this paper cites.
“Learning Transferable Visual Models From Natural Language Supervision”, 2021
Original
Alec Radford et al · 2021
Earlier work this paper cites.

Similar

“LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs”, 2021
Original
Christoph Schuhmann et al · 2021
Cited alongside, same era.
“Flamingo: a Visual Language Model for Few-Shot Learning”, 2022
Original
Jean-Baptiste Alayrac et al · 2022
Cited alongside, same era.
“BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation”, 2022
Original
Junnan Li, Dongxu Li, Caiming Xiong and Steven Hoi · 2022
Cited alongside, same era.
“Training language models to follow instructions with human feedback”, 2022
Original
Long Ouyang et al · 2022
Cited alongside, same era.
“LAION-5B: An open large-scale dataset for training next generation image-text models”, 2022

Then

“IC3: Image Captioning by Committee Consensus”, 2023
Original
David. Chan et al · 2023
Later among the works it cites.
“Deep reinforcement learning from human preferences”, 2023
Original
Paul Christiano et al · 2023
Later among the works it cites.
“BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models”, 2023
Original
Junnan Li, Dongxu Li, Silvio Savarese and Steven Hoi · 2023
Later among the works it cites.
“Llama 2: Open Foundation and Fine-Tuned Chat Models”, 2023
Original
Hugo Touvron et al · 2023
Later among the works it cites.
“LLaMA: Open and Efficient Foundation Language Models”, 2023
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Christoph Schuhmann et al · 2022

Cited alongside, same era.

“CoCa: Contrastive Captioners are Image-Text Foundation Models”, 2022

Jiahui Yu et al · 2022

Cited alongside, same era.

“Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language”, 2022

Andy Zeng et al · 2022

Cited alongside, same era.

“OPT: Open Pre-trained Transformer Language Models”, 2022

Susan Zhang et al · 2022

Cited alongside, same era.

Hugo Touvron et al · 2023

Later among the works it cites.

“Image Captioners Are Scalable Vision Learners Too”, 2023

Michael Tschannen et al · 2023

Later among the works it cites.

“ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions”, 2023

Deyao Zhu et al · 2023

Later among the works it cites.