CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects

N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J.-Y. Zhu, “Multi-concept customization of text-to-image diffusion,” in IEEE/CVF Computer Vision and Pattern Recognition Conference , 2023, pp. 1931–1941

1941

Earlier work this paper cites.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems , 2014, pp. 2672–2680

2014

Earlier work this paper cites.

A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” in Advances in neural information processing systems , 2017, pp. 6309–6318

2017

Earlier work this paper cites.

I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101 , 2017

Original

2017

Earlier work this paper cites.

Y. Balaji, M. R. Min, B. Bai, R. Chellappa, and H. P. Graf, “Conditional gan with discriminative filter generation for text-to-video synthesis,” in International Joint Conference on Artificial Intelligence , vol. 1, no. 2019, 2019, p. 2

2019

Earlier work this paper cites.

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems , vol. 32, 2019

2019

Earlier work this paper cites.

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840–6851, 2020

2020

Earlier work this paper cites.

J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” International Conference on Learning Representations , 2020

2020

Earlier work this paper cites.

J. Ho and T. Salimans, “Classifier-free diffusion guidance,” in NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications , 2021

2021

Earlier work this paper cites.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning , 2021, pp. 8748–8763

2021

Earlier work this paper cites.

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations , 2021

2021

Earlier work this paper cites.

M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in International Conference on Computer Vision , 2021, pp. 9650–9660

2021

Earlier work this paper cites.

C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,” Advances in neural information processing systems , vol. 35, pp. 5775–5787, 2022

2022

Earlier work this paper cites.

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in IEEE/CVF Computer Vision and Pattern Recognition Conference , 2022, pp. 10 684–10 695

2022

Earlier work this paper cites.

I. Skorokhodov, S. Tulyakov, and M. Elhoseiny, “Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2,” in IEEE/CVF Computer Vision and Pattern Recognition Conference , 2022, pp. 3626–3636

2022

Earlier work this paper cites.

Y. He, T. Yang, Y. Zhang, Y. Shan, and Q. Chen, “Latent video diffusion models for high-fidelity video generation with arbitrary lengths,” arXiv preprint arXiv:2211.13221 , 2022

Original

2022

Earlier work this paper cites.

E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations , 2022

2022

Earlier work this paper cites.

P. von Platen, S. Patil, A. Lozhkov, P. Cuenca, N. Lambert, K. Rasul, M. Davaadorj, and T. Wolf, “Diffusers: State-of-the-art diffusion models,” https://github.com/huggingface/diffusers , 2022

2022

Earlier work this paper cites.

S. Gugger, L. Debut, T. Wolf, P. Schmid, Z. Mueller, S. Mangrulkar, M. Sun, and B. Bossan, “Accelerate: Training and inference at scale made simple, efficient and adaptable,” https://github.com/huggingface/accelerate , 2022

2022

Earlier work this paper cites.

J. Wang, H. Yuan, D. Chen, Y. Zhang, X. Wang, and S. Zhang, “Modelscope text-to-video technical report,” arXiv preprint arXiv:2308.06571 , 2023

Original

2023

Earlier work this paper cites.

A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts et al. , “Stable video diffusion: Scaling latent video diffusion models to large datasets,” arXiv preprint arXiv:2311.15127 , 2023

Original

2023

Earlier work this paper cites.

H. Zhao, T. Lu, J. Gu, X. Zhang, Z. Wu, H. Xu, and Y.-G. Jiang, “Videoassembler: Identity-consistent video generation with reference entities using diffusion model,” arXiv preprint arXiv:2311.17338 , 2023

Original

2023

Earlier work this paper cites.

H. Chen, X. Wang, G. Zeng, Y. Zhang, Y. Zhou, F. Han, and W. Zhu, “Videodreamer: Customized multi-subject text-to-video generation with disen-mix finetuning,” arXiv preprint arXiv:2311.00990 , 2023

Original

2023

Earlier work this paper cites.

A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, P. Dollár, and R. Girshick, “Segment anything,” International Conference on Computer Vision , 2023

2023

Cited alongside, same era.

H. Chen, M. Xia, Y. He, Y. Zhang, X. Cun, S. Yang, J. Xing, Y. Liu, Q. Chen, X. Wang et al. , “Videocrafter1: Open diffusion models for high-quality video generation,” arXiv preprint arXiv:2310.19512 , 2023

Original

2023

Cited alongside, same era.

Z. Duan, L. You, C. Wang, C. Chen, Z. Wu, W. Qian, J. Huang, F. Chao, and R. Ji, “Diffsynth: Latent in-iteration deflickering for realistic video synthesis,” arXiv preprint arXiv:2308.03463 , 2023

Original

2023

Cited alongside, same era.

W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang, “Cogvideo: Large-scale pretraining for text-to-video generation via transformers,” International Conference on Learning Representations , 2023

2023

Cited alongside, same era.

G. Xiao, T. Yin, W. T. Freeman, F. Durand, and S. Han, “Fastcomposer: Tuning-free multi-subject image generation with localized attention,” arXiv preprint arXiv:2305.10431 , 2023

Original

2023

Later among the works it cites.

S. Sterling, “Zeroscope,” https://huggingface.co/cerspense/zeroscope_v2_576w , 2023

2023

Later among the works it cites.

Y. Gu, X. Wang, J. Z. Wu, Y. Shi, Y. Chen, Z. Fan, W. Xiao, R. Zhao, S. Chang, W. Wu et al. , “Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models,” Advances in Neural Information Processing Systems , vol. 36, 2023

2023

Later among the works it cites.

S. Sterling, “Zeroscope xl,” https://huggingface.co/cerspense/zeroscope_v2_XL , 2023

2023

Later among the works it cites.

P. Esser, J. Chiu, P. Atighehchian, J. Granskog, and A. Germanidis, “Structure and content-guided video synthesis with diffusion models,” in International Conference on Computer Vision , 2023, pp. 7346–7356