2021

Transformer for Single Image Super-Resolution

Lu, Zhisheng, Li, Juncheng, Liu, Hong et al.

Understand

Single image super-resolution (SISR) has witnessed great strides with the development of deep learning.

However, most existing studies focus on building more complex networks with a massive number of layers.
Recently, more and more researchers start to explore the application of Transformer in computer vision tasks.
However, the heavy computational cost and high GPU memory occupation of the vision Transformer cannot be ignored.

Built on

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics
David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik · 2001
Earlier work this paper cites.
Image super-resolution via sparse representation
Jianchao Yang, John Wright, Thomas S Huang, and Yi Ma · 2010
Earlier work this paper cites.
Low-complexity single-image super-resolution based on nonnegative neighbor embedding
Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel · 2012
Earlier work this paper cites.
Image super-resolution using deep convolutional networks
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang · 2015
Earlier work this paper cites.
Single image super-resolution from transformed self-exemplars
Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja · 2015
Earlier work this paper cites.
Layer normalization
Original
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Accurate image super-resolution using very deep convolutional networks
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee · 2016
Earlier work this paper cites.
Deeply-recursive convolutional network for image super-resolution
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee · 2016
Earlier work this paper cites.
Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang · 2016
Earlier work this paper cites.
Ntire 2017 challenge on single image super-resolution: Dataset and study
Eirikur Agustsson and Radu Timofte · 2017
Earlier work this paper cites.
Enhanced deep residual networks for single image super-resolution
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee · 2017
Earlier work this paper cites.
Image super-resolution via deep recursive residual network
Ying Tai, Jian Yang, and Xiaoming Liu · 2017
Earlier work this paper cites.
Memnet: A persistent memory network for image restoration
Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.

Similar

Fast, accurate, and lightweight super-resolution with cascading residual network
N. Ahn, B. Kang, and K. A. Sohn · 2018
Cited alongside, same era.
Cascaded deep networks with multiple receptive fields for infrared image super-resolution
Zewei He, Siliang Tang, Jiangxin Yang, Yanlong Cao, Michael Ying Yang, and Yanpeng Cao · 2018
Cited alongside, same era.
Squeeze-and-excitation networks
Jie Hu, Li Shen, and Gang Sun · 2018
Cited alongside, same era.
Fast and accurate single image super-resolution via information distillation network
Zheng Hui, Xiumei Wang, and Xinbo Gao · 2018
Cited alongside, same era.
Fast and accurate image super-resolution with deep laplacian pyramid networks
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang · 2018
Cited alongside, same era.
Multi-scale residual network for image super-resolution
Juncheng Li, Faming Fang, Kangfu Mei, and Guixu Zhang · 2018
Cited alongside, same era.
Ntire 2018 challenge on single image super-resolution: Methods and results
Radu Timofte, Shuhang Gu, Jiqing Wu, and Luc Van Gool · 2018
Cited alongside, same era.
Resolution-aware network for image super-resolution
Yifan Wang, Lijun Wang, Hongyu Wang, and Peihua Li · 2018
Cited alongside, same era.
Learning a single convolutional super-resolution network for multiple degradations
Kai Zhang, Wangmeng Zuo, and Lei Zhang · 2018
Cited alongside, same era.
Image super-resolution using very deep residual channel attention networks
Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu · 2018
Cited alongside, same era.
Residual dense network for image super-resolution
Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu · 2018
Cited alongside, same era.
Toward real-world single image super-resolution: A new benchmark and a new model
Jianrui Cai, Hui Zeng, Hongwei Yong, Zisheng Cao, and Lei Zhang · 2019
Cited alongside, same era.

Then

Building a manga dataset “manga109” with annotations for multimedia applications
Kiyoharu Aizawa, Azuma Fujimoto, Atsushi Otsubo, Toru Ogawa, Yusuke Matsui, Koki Tsubota, and Hikaru Ikuta · 2020
Later among the works it cites.
End-to-end object detection with transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko · 2020
Later among the works it cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Original
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Later among the works it cites.
Residual feature distillation network for lightweight image super-resolution
Jie Liu, Jie Tang, and Gangshan Wu · 2020
Later among the works it cites.
Latticenet: Towards lightweight image super-resolution with lattice block
Xiaotong Luo, Yuan Xie, Yulun Zhang, Yanyun Qu, Cuihua Li, and Yun Fu · 2020
Later among the works it cites.
Multi-attention based ultra lightweight image super-resolution
Abdul Muqeet, Jiwon Hwang, Subin Yang, JungHeum Kang, Yongwoo Kim, and Sung-Ho Bae · 2020
Later among the works it cites.
Learning texture transformer network for image super-resolution
Fuzhi Yang, Huan Yang, Jianlong Fu, Hongtao Lu, and Baining Guo · 2020
Later among the works it cites.
Deformable detr: Deformable transformers for end-to-end object detection
Original
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai · 2020
Later among the works it cites.
Pre-trained image processing transformer
Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao · 2021
Closest in time.
From beginner to master: A survey for deep learning-based single-image super-resolution
Original
Juncheng Li, Zehua Pei, and Tieyong Zeng · 2021
Closest in time.
Localvit: Bringing locality to vision transformers
Original
Yawei Li, Kai Zhang, Jiezhang Cao, Radu Timofte, and Luc Van Gool · 2021
Closest in time.
Swinir: Image restoration using swin transformer
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte · 2021
Closest in time.
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo · 2021
Closest in time.
Training data-efficient image transformers & distillation through attention
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou · 2021
Closest in time.
Aligned structured sparsity learning for efficient image super-resolution
Yulun Zhang, Huan Wang, Can Qin, and Yun Fu · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…