2018

Learning Image Representations by Completing Damaged Jigsaw Puzzles

Kim, Dahun, Cho, Donghyeon, Yoo, Donggeun et al.

Understand

In this paper, we explore methods of complicating self-supervised tasks for representation learning.

That is, we do severe damage to data and encourage a network to recover them.
First, we complicate each of three powerful self-supervised task candidates: jigsaw puzzle, inpainting, and colorization.
In addition, we introduce a novel complicated self-supervised task called "Completing damaged jigsaw puzzles" which is puzzles with one piece missing and the other pieces without color.

Built on

The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman · 2007
Earlier work this paper cites.
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol · 2010
Earlier work this paper cites.
The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Auto-encoding variational bayes
M. W. Diederik P Kingma · 2014
Earlier work this paper cites.
Caffe: Convolutional architecture for fast feature embedding
Original
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Original
K. Simonyan and A. Zisserman · 2014
Earlier work this paper cites.
Learning to see by moving
P. Agrawal, J. Carreira, and J. Malik · 2015
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
C. Doersch, A. Gupta, and A. A. Efros · 2015
Earlier work this paper cites.
Fast r-cnn
R. Girshick · 2015
Earlier work this paper cites.
Learning visual groups from co-occurrences in space and time
P. Isola, D. Zoran, D. Krishnan, and E. H. Adelson · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
D. P. Kingma and J. Ba · 2015
Earlier work this paper cites.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Earlier work this paper cites.

Similar

Is object localization for free? - weakly-supervised learning with convolutional neural networks
M. Oquab, L. Bottou, I. Laptev, and J. Sivic · 2015
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei · 2015
Cited alongside, same era.
Going deeper with convolutions
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich · 2015
Cited alongside, same era.
Unsupervised learning of visual representations using videos
X. Wang and A. Gupta · 2015
Cited alongside, same era.
Weakly supervised deep detection networks
H.Bilen and A. Vedaldi · 2016
Cited alongside, same era.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Contextlocnet: Context-aware deep network models for weakly supervised localization
V. Kantorov, M. Oquab, M. Cho, and I. Laptev · 2016
Cited alongside, same era.
Data-dependent initializations of convolutional neural networks
P. Krähenbühl, C. Doersch, J. Donahue, and T. Darrell · 2016
Cited alongside, same era.
Shuffle and learn: Unsupervised learning using temporal order verification
I. Misra, C. L. Zitnick, and M. Hebert · 2016
Cited alongside, same era.
Unsupervised visual representation learning by context prediction
M. Noroozi and P. Favaro · 2016
Cited alongside, same era.
Look, listen and learn
A. Owens, J. Wu, J. H. McDermott, W. T. Freeman, and A. Torralba · 2016
Cited alongside, same era.
Context encoders: Feature learning by inpainting
D. Pathak, P. Krähenbühl, J. Donahue, T. Darrell, and A. A. Efros · 2016
Cited alongside, same era.

Then

Colorful image colorization
R. Zhang, P. Isola, and A. A. Efros · 2016
Later among the works it cites.
Ambient sound provides supervision for visual learning
R. Arandjelovic and A. Zisserman · 2017
Later among the works it cites.
Multi-task self-supervised visual learning
C. Doersch and A. Zisserman · 2017
Later among the works it cites.
Adversarial feature learning
J. Donahue, P. Krähenbühl, and T. Darrelln · 2017
Later among the works it cites.
Learning image representations tied to egomotion from unlabeled video
D. Jayaraman and K. Grauman · 2017
Later among the works it cites.
Deep self-taught learning for weakly supervised object localization
Z. Jie, Y. Wei, X. Jin, J. Feng, and W. Liu · 2017
Later among the works it cites.
Two-phase learning for weakly supervised object localization
D. Kim, D. Cho, and D. Yoo · 2017
Later among the works it cites.
Colorization as a proxy task for visual understanding
G. Larsson, M. Maire, and G. Shakhnarovich · 2017
Later among the works it cites.
Unsupervised representation learning by sorting sequences
H.-Y. Lee, J.-B. Huang, M. Singh, and M.-H. Yang · 2017
Later among the works it cites.
Representation learning by learning to count
M. Noroozi, H. Pirsiavash, and P. Favaro · 2017
Later among the works it cites.
Learning features by watching objects move
D. Pathak, R. B. Girshick, P. Dollár, T. Darrell, and B. Hariharan · 2017
Later among the works it cites.
Transitive invariance for self-supervised visual representation learning
X. Wang, K. He, and A. Gupta · 2017
Later among the works it cites.
Split-brain autoencoders: Unsupervised learning by cross-channel prediction
R. Zhang, P. Isola, and A. A. Efros · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…