Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation

Difftalk: Crafting diffusion models for generalized audio-driven portraits animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 1982–1991

Shuai Shen, Wenliang Zhao, Zibin Meng, Wanhua Li, Zheng Zhu, Jie Zhou, and Jiwen Lu. 2023 · 1991

Earlier work this paper cites.

Face recognition based on fitting a 3D morphable model

Volker Blanz and Thomas Vetter. 2003 · 2003

Earlier work this paper cites.

Learning a model of facial shape and expression from 4D scans

Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. 2017 · 2017

Earlier work this paper cites.

Neural discrete representation learning

Aaron Van Den Oord, Oriol Vinyals, et al · 2017

Earlier work this paper cites.

wav2vec: Unsupervised pre-training for speech recognition

Original

Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019 · 2019

Earlier work this paper cites.

First order motion model for image animation

Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. 2019 · 2019

Earlier work this paper cites.

Neural head reenactment with latent pose descriptors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 13786–13795

Egor Burkov, Igor Pasechnik, Artur Grigorev, and Victor Lempitsky. 2020 · 2020

Earlier work this paper cites.

A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM international conference on multimedia (ACM MM) . 484–492

KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Namboodiri, and CV Jawahar. 2020 · 2020

Earlier work this paper cites.

Fast bi-layer neural synthesis of one-shot realistic head avatars. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16 . Springer, 524–540

Egor Zakharov, Aleksei Ivakhnenko, Aliaksandra Shysheya, and Victor Lempitsky. 2020 · 2020

Earlier work this paper cites.

Learning transferable visual models from natural language supervision. In International conference on machine learning (ICML) . 8748–8763

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021

Earlier work this paper cites.

Pirenderer: Controllable portrait image generation via semantic neural rendering. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 13759–13768

Yurui Ren, Ge Li, Yuanqi Chen, Thomas H Li, and Shan Liu. 2021 · 2021

Earlier work this paper cites.

Pose-controllable talking face generation by implicitly modularized audio-visual representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 4176–4186

Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, and Ziwei Liu. 2021 · 2021

Earlier work this paper cites.

Megaportraits: One-shot megapixel neural head avatars. In Proceedings of the 30th ACM International Conference on Multimedia (ACM MM) . 2663–2671

Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Aleksei Ivakhnenko, Victor Lempitsky, and Egor Zakharov. 2022 · 2022

Earlier work this paper cites.

Flexible diffusion modeling of long videos

William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood. 2022 · 2022

Earlier work this paper cites.

Imagen video: High definition video generation with diffusion models

Original

Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al · 2022

Earlier work this paper cites.

Video diffusion models

Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. 2022b · 2022

Earlier work this paper cites.

Headnerf: A real-time nerf-based parametric head model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 20374–20384

Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, and Juyong Zhang. 2022 · 2022

Cited alongside, same era.

Expressive talking head generation with granular audio-visual control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 3387–3396

Borong Liang, Yan Pan, Zhizhi Guo, Hang Zhou, Zhibin Hong, Xiaoguang Han, Junyu Han, Jingtuo Liu, Errui Ding, and Jingdong Wang. 2022 · 2022

Cited alongside, same era.

Make-A-Video: Text-to-Video Generation without Text-Video Data. In The Eleventh International Conference on Learning Representations (ICLR)

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al · 2022

Cited alongside, same era.

Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan. In European Conference on Computer Vision (ECCV) . Springer, 85–101

Fei Yin, Yong Zhang, Xiaodong Cun, Mingdeng Cao, Yanbo Fan, Xuan Wang, Qingyan Bai, Baoyuan Wu, Jue Wang, and Yujiu Yang. 2022 · 2022

Dreamtalk: When expressive talking head generation meets diffusion probabilistic models

Original

Yifeng Ma, Shiwei Zhang, Jiayu Wang, Xiang Wang, Yingya Zhang, and Zhidong Deng. 2023 · 2023

Later among the works it cites.

Dreamix: Video diffusion models are general video editors

Original

Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen. 2023 · 2023

Later among the works it cites.

Dpe: Disentanglement of pose and expression for general video portrait editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 427–436

Youxin Pang, Yong Zhang, Weize Quan, Yanbo Fan, Xiaodong Cun, Ying Shan, and Dong-ming Yan. 2023 · 2023

Later among the works it cites.

Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 4195–4205

William Peebles and Saining Xie. 2023 · 2023

Later among the works it cites.