Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching

A family of embedded runge-kutta formulae

John R Dormand and Peter J Prince · 1980

Earlier work this paper cites.

Signal estimation from modified short-time fourier transform

Daniel Griffin and Jae Lim · 1984

Earlier work this paper cites.

Audio set: An ontology and human-labeled dataset for audio events

Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter · 2017

Earlier work this paper cites.

Improving language understanding by generative pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018

Earlier work this paper cites.

Vggsound: A large-scale audio-visual dataset

Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman · 2020

Earlier work this paper cites.

Denoising diffusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel · 2020

Earlier work this paper cites.

Taming visually guided sound generation

Vladimir Iashin and Esa Rahtu · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021

Earlier work this paper cites.

Foleygan: Visually guided generative adversarial network-based synchronous sound generation in silent videos

Sanchita Ghose and John J Prevost · 2022

Earlier work this paper cites.

Bigvgan: A universal neural vocoder with large-scale training

Sang-gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, and Sungroh Yoon · 2022

Earlier work this paper cites.

Flow matching for generative modeling

Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le · 2022

Earlier work this paper cites.

Flow straight and fast: Learning to generate and transfer data with rectified flow

Xingchao Liu, Chengyue Gong, et al · 2022

Cited alongside, same era.

Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu · 2022

Cited alongside, same era.

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer · 2022

Cited alongside, same era.

Photorealistic text-to-image diffusion models with deep language understanding

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al · 2022

Cited alongside, same era.

Make-a-video: Text-to-video generation without text-video data

Original

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al · 2022

Cited alongside, same era.

Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation

Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov · 2023

Later among the works it cites.

Scaling rectified flow transformers for high-resolution image synthesis

Original

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al · 2024

Closest in time.

Voiceflow: Efficient text-to-speech with rectified flow matching

Yiwei Guo, Chenpeng Du, Ziyang Ma, Xie Chen, and Kai Yu · 2024

Closest in time.

Mavil: Masked audio-video learners

Po-Yao Huang, Vasu Sharma, Hu Xu, Chaitanya Ryali, Yanghao Li, Shang-Wen Li, Gargi Ghosh, Jitendra Malik, Christoph Feichtenhofer, et al · 2024

Closest in time.

Voicebox: Text-guided multilingual universal speech generation at scale

Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al · 2024