2018

Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency

Ma, Liqian, Jia, Xu, Georgoulis, Stamatios et al.

Understand

Image-to-image translation has recently received significant attention due to advances in deep learning.

Most works focus on learning either a one-to-one mapping in an unsupervised way or a many-to-many mapping in a supervised way.
However, a more practical setting is many-to-many mapping in an unsupervised way, which is harder due to the lack of supervision and the complex inner- and cross-domain variations.
To alleviate these issues, we propose the Exemplar Guided & Semantically Consistent Image-to-image Translation (EGSC-IT) network which conditions the translation process on an exemplar image in the target domain.

Built on

Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Visualizing data using t-sne
Laurens van der Maaten and Geoffrey Hinton · 2008
Earlier work this paper cites.
Auto-encoding variational bayes
Original
Diederik P Kingma and Max Welling · 2013
Earlier work this paper cites.
Learning a deep convolutional network for image super-resolution
Chao Dong, Chen C. Loy, Kaiming He, and Xiaoou Tang · 2014
Earlier work this paper cites.
Generative adversarial nets
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Image style transfer using convolutional neural networks
Leon A Gatys, Alexander S Ecker, and Matthias Bethge · 2016
Earlier work this paper cites.
Perceptual losses for real-time style transfer and super-resolution
Justin Johnson, Alexandre Alahi, and Li Fei-Fei · 2016
Earlier work this paper cites.
Autoencoding beyond pixels using a learned similarity metric
Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther · 2016
Earlier work this paper cites.
Context encoders: Feature learning by inpainting
Deepak Pathak, Philipp Krähenbühl, Jeff Donahue, Trevor Darrell, and Alexei A. Efros · 2016
Earlier work this paper cites.
Playing for data: Ground truth from computer games
Stephan R. Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun · 2016
Earlier work this paper cites.

Similar

Colorful image colorization
Richard Zhang, Phillip Isola, and Alexei A. Efros · 2016
Cited alongside, same era.
Controlling perceptual factors in neural style transfer
Leon A Gatys, Alexander S Ecker, Matthias Bethge, Aaron Hertzmann, and Eli Shechtman · 2017
Cited alongside, same era.
Arbitrary style transfer in real-time with adaptive instance normalization
Xun Huang and Serge J. Belongie · 2017
Cited alongside, same era.
Image-to-image translation with conditional adversarial networks
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros · 2017
Cited alongside, same era.
Learning to discover cross-domain relations with generative adversarial networks
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim · 2017
Cited alongside, same era.
Universal style transfer via feature transforms
Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang · 2017
Cited alongside, same era.
Unsupervised image-to-image translation networks
Ming-Yu Liu, Thomas Breuel, and Jan Kautz · 2017
Cited alongside, same era.
Deep photo style transfer
Fujun Luan, Sylvain Paris, Eli Shechtman, and Kavita Bala · 2017
Cited alongside, same era.
Xgan: Unsupervised image-to-image translation for many-to-many mappings
Original
Amélie Royer, Konstantinos Bousmalis, Stephan Gouws, Fred Bertsch, Inbar Moressi, Forrester Cole, and Kevin Murphy · 2017
Cited alongside, same era.
End-to-end learning of driving models from large-scale video datasets
Huazhe Xu, Yang Gao, Fisher Yu, and Trevor Darrell · 2017
Cited alongside, same era.
Unpaired image-to-image translation using cycle-consistent adversarial networks
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros
Cited in the paper.
Toward multimodal image-to-image translation
Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman
Cited in the paper.

Then

Dualgan: Unsupervised dual learning for image-to-image translation
Zili Yi, Hao (Richard) Zhang, Ping Tan, and Minglun Gong · 2017
Later among the works it cites.
Augmented cyclegan: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, and Aaron Courville · 2018
Closest in time.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille · 2018
Closest in time.
Image-to-image translation for cross-domain disentanglement
Abel Gonzalez-Garcia, Joost van de Weijer, and Yoshua Bengio · 2018
Closest in time.
Cycada: Cycle-consistent adversarial domain adaptation
Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell · 2018
Closest in time.
Multimodal unsupervised image-to-image translation
Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz · 2018
Closest in time.
Diverse image-to-image translation via disentangled representations
Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang · 2018
Closest in time.
A closed-form solution to photorealistic image stylization
Original
Yijun Li, Ming-Yu Liu, Xueting Li, Ming-Hsuan Yang, and Jan Kautz · 2018
Closest in time.
Conditional image-to-image translation
Jianxin Lin, Yingce Xia, Tao Qin, Zhibo Chen, and Tie-Yan Liu · 2018
Closest in time.
High-resolution image synthesis and semantic manipulation with conditional gans
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…