2017

Deep Photo Style Transfer

Luan, Fujun, Paris, Sylvain, Shechtman, Eli et al.

Understand

This paper introduces a deep-learning approach to photographic style transfer that handles a large variety of image content while faithfully transferring the reference style.

Our approach builds upon the recent work on painterly transfer that separates style from the content of an image by considering different layers of a neural network.
However, as is, this approach is not suitable for photorealistic style transfer.
Even when both the input and reference images are photographs, the output still exhibits distortions reminiscent of a painting.

Built on

Image analogies
A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin · 2001
Earlier work this paper cites.
Color transfer between images
E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley · 2001
Earlier work this paper cites.
N-dimensional probability density function transfer and its application to color transfer
F. Pitie, A. C. Kokaram, and R. Dahyot · 2005
Earlier work this paper cites.
Two-scale tone management for photographic look
S. Bae, S. Paris, and F. Durand · 2006
Earlier work this paper cites.
A closed-form solution to natural image matting
A. Levin, D. Lischinski, and Y. Weiss · 2008
Earlier work this paper cites.
Multi-scale image harmonization
K. Sunkavalli, M. K. Johnson, W. Matusik, and H. Pfister · 2010
Earlier work this paper cites.

Similar

Data-driven hallucination of different times of day from a single outdoor photo
Y. Shih, S. Paris, F. Durand, and W. T. Freeman · 2013
Cited alongside, same era.
Content-based colour transfer
F. Wu, W. Dong, Y. Kong, X. Mei, J.-C. Paul, and X. Zhang · 2013
Cited alongside, same era.
Transient attributes for high-level understanding and editing of outdoor scenes
P.-Y. Laffont, Z. Ren, X. Tao, C. Qian, and J. Hays · 2014
Cited alongside, same era.
Style transfer for headshot portraits
Y. Shih, S. Paris, C. Barnes, W. T. Freeman, and F. Durand · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
Original
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Deep manifold traversal: Changing labels with convolutional features
Original
J. R. Gardner, M. J. Kusner, Y. Li, P. Upchurch, K. Q. Weinberger, K. Bala, and J. E. Hopcroft · 2015
Cited alongside, same era.
Gram matrix
E. W. Weisstein
Cited in the paper.

Then

neural-style
J. Johnson · 2015
Later among the works it cites.
Semantic style transfer and turning two-bit doodles into fine artworks
A. J. Champandard · 2016
Later among the works it cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Original
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2016
Later among the works it cites.
Image style transfer using convolutional neural networks
L. A. Gatys, A. S. Ecker, and M. Bethge · 2016
Later among the works it cites.
Combining markov random fields and convolutional neural networks for image synthesis
Original
C. Li and M. Wand · 2016
Later among the works it cites.
Painting style transfer for head portraits using convolutional neural networks
A. Selim, M. Elgharib, and L. Doyle · 2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…