Fetching the paper…
Reading the bibliography…
Diffusion Transformers (DiTs) are essential for video generation but suffer from significant latency due to the quadratic complexity of attention.
k-means++: the advantages of careful seeding
David Arthur and Sergei Vassilvitskii · 2007
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Transformers are rnns: Fast autoregressive transformers with linear attention, 2020
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret · 2020
Earlier work this paper cites.
Nvidia a100 tensor core gpu
NVIDIA · 2021
Earlier work this paper cites.
H2o: Heavy-hitter oracle for efficient generative inference of large language models, 2023
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen · 2023
Earlier work this paper cites.
Pit: Optimization of dynamic sparse deep learning models via permutation invariant transformation, 2023
Ningxin Zheng, Huiqiang Jiang, Quanlu Zhang, Zhenhua Han, Yuqing Yang, Lingxiao Ma, Fan Yang, Chengruidong Zhang, Lili Qiu, Mao Yang, and Lidong Zhou · 2023
Earlier work this paper cites.
Sparsetir: Composable abstractions for sparse compilation in deep learning, 2023
Zihao Ye, Ruihang Lai, Junru Shao, Tianqi Chen, and Luis Ceze · 2023
Earlier work this paper cites.
Vbench: Comprehensive benchmark suite for video generative models, 2023
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu · 2023
Earlier work this paper cites.
Hunyuanvideo: A systematic framework for large video generative models
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al · 2024
Earlier work this paper cites.
Cogvideox: Text-to-video diffusion models with an expert transformer
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al · 2024
Earlier work this paper cites.
Quest: Query-aware sparsity for efficient long-context llm inference
Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han · 2024
Earlier work this paper cites.
Efficient streaming language models with attention sinks
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis · 2024
Earlier work this paper cites.
Post-training sparse attention with double sparsity, 2024
Shuo Yang, Ying Sheng, Joseph E. Gonzalez, Ion Stoica, and Lianmin Zheng · 2024
Earlier work this paper cites.
Duoattention: Efficient long-context llm inference with retrieval and streaming heads, 2024
Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, and Song Han · 2024
Earlier work this paper cites.
Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention
Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir Abdi, Dongsheng Li, Chin-Yew Lin, et al · 2024
Earlier work this paper cites.
Infllm: Training-free long-context extrapolation for llms with an efficient context memory, 2024
Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, and Maosong Sun · 2024
Earlier work this paper cites.
Lm-infinite: Zero-shot extreme length generalization for large language models, 2024
Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang · 2024
Earlier work this paper cites.
Gated linear attention transformers with hardware-efficient training, 2024
Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim · 2024
Earlier work this paper cites.
Mamba: Linear-time sequence modeling with selective state spaces, 2024
Albert Gu and Tri Dao · 2024
Earlier work this paper cites.
Transformers are ssms: Generalized models and efficient algorithms through structured state space duality, 2024
Tri Dao and Albert Gu · 2024
Earlier work this paper cites.
Matten: Video generation with mamba-attention, 2024
Yu Gao, Jiancheng Huang, Xiaopeng Sun, Zequn Jie, Yujie Zhong, and Lin Ma · 2024
Earlier work this paper cites.
Sana: Efficient high-resolution image synthesis with linear diffusion transformers, 2024
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han · 2024
Earlier work this paper cites.
Fastercache: Training-free video diffusion model acceleration with high quality
Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K. Wong · 2024
Earlier work this paper cites.
Flashattention-3: Fast and accurate attention with asynchrony and low-precision
Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao · 2024
Cited alongside, same era.
Flex attention: A programming model for generating optimized attention kernels, 2024
Juechu Dong, Boyuan Feng, Driss Guessous, Yanbo Liang, and Horace He · 2024
Cited alongside, same era.
Distrifusion: Distributed parallel inference for high-resolution diffusion models
Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Ming-Yu Liu, Kai Li, and Song Han · 2024
Cited alongside, same era.
Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity
Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al · 2025
Cited alongside, same era.
Fast video generation with sliding tile attention
Peiyuan Zhang, Yongqi Chen, Runlong Su, Hangliang Ding, Ion Stoica, Zhenghong Liu, and Hao Zhang · 2025
Long-context state-space video world models, 2025
Ryan Po, Yotam Nitzan, Richard Zhang, Berlin Chen, Tri Dao, Eli Shechtman, Gordon Wetzstein, and Xun Huang · 2025
Closest in time.
One-minute video generation with test-time training, 2025
Karan Dalal, Daniel Koceja, Gashon Hussein, Jiarui Xu, Yue Zhao, Youjin Song, Shihao Han, Ka Chun Cheung, Jan Kautz, Carlos Guestrin, Tatsunori Hashimoto, Sanmi Koyejo, Yejin Choi, Yu Sun, and Xiaolong Wang · 2025
Closest in time.
Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer, 2025
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, Bingchen Liu, Daquan Zhou, and Song Han · 2025
Closest in time.
Sana-video: Efficient video generation with block linear diffusion transformer, 2025
Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, Junyu Chen, Shuai Yang, Xianbang Wang, Yicheng Pan, Daquan Zhou, Huan Ling, Haozhe Liu, Hongwei Yi, Hao Zhang, Muyang Li, Yukang Chen, Han Cai, Sanja Fidler, Ping Luo, Song Han, and Enze Xie · 2025
Closest in time.
Dc-ae 1.5: Accelerating diffusion model convergence with structured latent space, 2025
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Spargeattn: Accurate sparse attention accelerating any model inference
Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen · 2025
Cited alongside, same era.
Xattention: Block sparse attention with antidiagonal scoring
Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, and Song Han · 2025
Cited alongside, same era.
Tactic: Adaptive sparse attention with clustering and distribution fitting for long-context llms, 2025
Kan Zhu, Tian Tang, Qinyu Xu, Yile Gu, Zhichen Zeng, Rohan Kadekodi, Liangyu Zhao, Ang Li, Arvind Krishnamurthy, and Baris Kasikci · 2025
Cited alongside, same era.
Mminference: Accelerating pre-filling for long-context vlms via modality-aware permutation sparse attention, 2025
Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, and Lili Qiu · 2025
Cited alongside, same era.
Twilight: Adaptive attention sparsity with hierarchical top-
Chaofan Lin, Jiaming Tang, Shuo Yang, Hanshuo Wang, Tian Tang, Boyu Tian, Ion Stoica, Song Han, and Mingyu Gao · 2025
Cited alongside, same era.
Flashinfer: Efficient and customizable attention engine for llm inference serving, 2025
Zihao Ye, Lequn Chen, Ruihang Lai, Wuwei Lin, Yineng Zhang, Stephanie Wang, Tianqi Chen, Baris Kasikci, Vinod Grover, Arvind Krishnamurthy, and Luis Ceze · 2025
Cited alongside, same era.
Training-free and adaptive sparse attention for efficient long video generation, 2025
Yifei Xia, Suhan Ling, Fangcheng Fu, Yujie Wang, Huixia Li, Xuefeng Xiao, and Bin Cui · 2025
Cited alongside, same era.
Junyu Chen, Dongyun Zou, Wenkun He, Junsong Chen, Enze Xie, Song Han, and Han Cai · 2025
Closest in time.
Dc-gen: Post-training diffusion acceleration with deeply compressed latent space, 2025
Wenkun He, Yuchao Gu, Junyu Chen, Dongyun Zou, Yujun Lin, Zhekai Zhang, Haocheng Xi, Muyang Li, Ligeng Zhu, Jincheng Yu, Junsong Chen, Enze Xie, Song Han, and Han Cai · 2025
Closest in time.
Deep compression autoencoder for efficient high-resolution diffusion models, 2025
Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han · 2025
Closest in time.
From slow bidirectional to fast autoregressive video diffusion models
Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang · 2025
Closest in time.
Self forcing: Bridging the train-test gap in autoregressive video diffusion, 2025
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman · 2025
Closest in time.
Longlive: Real-time interactive long video generation, 2025
Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, Song Han, and Yukang Chen · 2025
Closest in time.
Packing input frame context in next-frame prediction models for video generation, 2025
Lvmin Zhang and Maneesh Agrawala · 2025
Closest in time.
Riflex: A free lunch for length extrapolation in video diffusion transformers, 2025
Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, and Jun Zhu · 2025
Closest in time.
Freelong++: Training-free long video generation via multi-band spectralfusion, 2025
Yu Lu and Yi Yang · 2025
Closest in time.
Radial attention:
Xingyang Li, Muyang Li, Tianle Cai, Haocheng Xi, Shuo Yang, Yujun Lin, Lvmin Zhang, Songlin Yang, Jinbo Hu, Kelly Peng, Maneesh Agrawala, Ion Stoica, Kurt Keutzer, and Song Han · 2025
Closest in time.
Vmoba: Mixture-of-block attention for video diffusion models, 2025
Jianzong Wu, Liang Hou, Haotian Yang, Xin Tao, Ye Tian, Pengfei Wan, Di Zhang, and Yunhai Tong · 2025
Closest in time.
Mixture of contexts for long video generation, 2025
Shengqu Cai, Ceyuan Yang, Lvmin Zhang, Yuwei Guo, Junfei Xiao, Ziyan Yang, Yinghao Xu, Zhenheng Yang, Alan Yuille, Leonidas Guibas, Maneesh Agrawala, Lu Jiang, and Gordon Wetzstein · 2025
Closest in time.
Vsa: Faster video diffusion with trainable sparse attention, 2025
Peiyuan Zhang, Yongqi Chen, Haofeng Huang, Will Lin, Zhengzhong Liu, Ion Stoica, Eric Xing, and Hao Zhang · 2025
Closest in time.
Dicache: Let diffusion model determine its own cache, 2025
Jiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang, Yuhang Zang, Tong Wu, Dahua Lin, and Jiaqi Wang · 2025
Closest in time.
Omnicache: A trajectory-oriented global perspective on training-free cache reuse for diffusion transformer models, 2025
Huanpeng Chu, Wei Wu, Guanyu Fen, and Yutao Zhang · 2025
Closest in time.
Timestep embedding tells: It’s time to cache for video diffusion model
Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan · 2025
Closest in time.
Less is enough: Training-free video diffusion acceleration via runtime-adaptive caching, 2025
Xin Zhou, Dingkang Liang, Kaijin Chen, Tianrui Feng, Xiwu Chen, Hongkai Lin, Yikang Ding, Feiyang Tan, Hengshuang Zhao, and Xiang Bai · 2025
Closest in time.
Real-time video generation with pyramid attention broadcast, 2025
Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You · 2025
Closest in time.