Fetching the paper…
Reading the bibliography…
We present a new method for text-driven motion transfer - synthesizing a video that complies with an input text prompt describing the target objects and scene while maintaining an input video's motion and scene layout.
Motion magnification
Ce Liu, Antonio Torralba, William T Freeman, Frédo Durand, and Edward H Adelson · 2005
Earlier work this paper cites.
Deep inside convolutional networks: Visualising image classification models and saliency maps
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Understanding deep image representations by inverting them
Aravindh Mahendran and Andrea Vedaldi · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015
Earlier work this paper cites.
Deep unsupervised learning using nonequilibrium thermodynamics
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli · 2015
Earlier work this paper cites.
Realtime multi-person 2d pose estimation using part affinity fields
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh · 2017
Earlier work this paper cites.
Sync-draw: Automatic video generation using deep recurrent attentive architectures
Gaurav Mittal, Tanya Marwah, and Vineeth N Balasubramanian · 2017
Earlier work this paper cites.
To create what you tell: Generating videos from captions
Yingwei Pan, Zhaofan Qiu, Ting Yao, Houqiang Li, and Tao Mei · 2017
Earlier work this paper cites.
The 2017 davis challenge on video object segmentation
Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine-Hornung, and Luc Van Gool · 2017
Earlier work this paper cites.
Synthesizing images of humans in unseen poses
Guha Balakrishnan, Amy Zhao, Adrian V Dalca, Fredo Durand, and John Guttag · 2018
Earlier work this paper cites.
Deep video portraits
Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Niessner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, and Christian Theobalt · 2018
Earlier work this paper cites.
Video generation from text
Yitong Li, Martin Min, Dinghan Shen, David Carlson, and Lawrence Carin · 2018
Earlier work this paper cites.
X2face: A network for controlling face generation using images, audio, and pose codes
Olivia Wiles, A Koepke, and Andrew Zisserman · 2018
Earlier work this paper cites.
Everybody dance now
Caroline Chan, Shiry Ginosar, Tinghui Zhou, and Alexei A Efros · 2019
Earlier work this paper cites.
Denoising diffusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel · 2020
Earlier work this paper cites.
Denoising diffusion implicit models
Jiaming Song, Chenlin Meng, and Stefano Ermon · 2020
Earlier work this paper cites.
Diffusion models beat gans on image synthesis
Prafulla Dhariwal and Alexander Nichol · 2021
Earlier work this paper cites.
Layered neural atlases for consistent video editing
Yoni Kasten, Dolev Ofri, Oliver Wang, and Tali Dekel · 2021
Earlier work this paper cites.
Jokr: Joint keypoint representation for unsupervised cross-domain motion retargeting
Ron Mokady, Rotem Tzaban, Sagie Benaim, Amit H Bermano, and Daniel Cohen-Or · 2021
Earlier work this paper cites.
Improved denoising diffusion probabilistic models
Alexander Quinn Nichol and Prafulla Dhariwal · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Cited alongside, same era.
C2f-fwn: Coarse-to-fine flow warping network for spatial-temporal consistent motion transfer
Dongxu Wei, Xiaowei Xu, Haibin Shen, and Kejie Huang · 2021
Cited alongside, same era.
Text2live: Text-driven layered image and video editing
Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, and Tali Dekel · 2022
Cited alongside, same era.
Diffusion models in vision: A survey
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah · 2022
Cited alongside, same era.
Control-a-video: Controllable text-to-video generation with diffusion models, 2023
Weifeng Chen, Jie Wu, Pan Xie, Hefeng Wu, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin · 2023
Closest in time.
Diffusion self-guidance for controllable image generation
Dave Epstein, Allan Jabri, Ben Poole, Alexei A. Efros, and Aleksander Holynski · 2023
Closest in time.
Structure and content-guided video synthesis with diffusion models
Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis · 2023
Closest in time.
Preserve your own correlation: A noise prior for video diffusion models
Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji · 2023
Closest in time.
Tokenflow: Consistent diffusion features for consistent video editing
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or · 2022
Cited alongside, same era.
Imagen video: High definition video generation with diffusion models
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al · 2022
Cited alongside, same era.
Text-driven stylization of video objects
Sebastian Loeschcke, Serge Belongie, and Sagie Benaim · 2022
Cited alongside, same era.
SDEdit: Guided image synthesis and editing with stochastic differential equations
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon · 2022
Cited alongside, same era.
Make-a-video: Text-to-video generation without text-video data, 2022
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman · 2022
Cited alongside, same era.
Structure-aware motion transfer with deformable anchor model
Jiale Tao, Biao Wang, Borun Xu, Tiezheng Ge, Yuning Jiang, Wen Li, and Lixin Duan · 2022
Cited alongside, same era.
Splicing vit features for semantic appearance transfer
Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel · 2022
Cited alongside, same era.
Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel · 2023
Closest in time.
Normal-guided garment uv prediction for human re-texturing
Yasamin Jafarian, Tuanfeng Y Wang, Duygu Ceylan, Jimei Yang, Nathan Carr, Yi Zhou, and Hyun Soo Park · 2023
Closest in time.
Cotracker: It is better to track together
Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht · 2023
Closest in time.
Text2video-zero: Text-to-image diffusion models are zero-shot video generators
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi · 2023
Closest in time.
Shape-aware text-driven layered video editing
Yao-Chih Lee, Ji-Ze Genevieve Jang, Yi-Ting Chen, Elizabeth Qiu, and Jia-Bin Huang · 2023
Closest in time.
Directed diffusion: Direct control of object placement through attention guidance
Wan-Duo Kurt Ma, JP Lewis, W Bastiaan Kleijn, and Thomas Leung · 2023
Closest in time.
Dreamix: Video diffusion models are general video editors
Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen · 2023
Closest in time.
Dragondiffusion: Enabling drag-style manipulation on diffusion models
Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang · 2023
Closest in time.
Zero-shot image-to-image translation
Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu · 2023
Closest in time.
Localizing object-level shape variations with text-to-image diffusion models
Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch-Elor, and Daniel Cohen-Or · 2023
Closest in time.
State of the art on diffusion models for visual computing
Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T Barron, Amit H Bermano, Eric Ryan Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, et al · 2023
Closest in time.
Fatezero: Fusing attentions for zero-shot text-based video editing
Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen · 2023
Closest in time.
Emergent correspondence from image diffusion
Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan · 2023
Closest in time.
Plug-and-play diffusion features for text-driven image-to image translation
Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel · 2023
Closest in time.
Restart sampling for improving generative processes
Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi S. Jaakkola · 2023
Closest in time.