2016

Deep Identity-aware Transfer of Facial Attributes

Li, Mu, Zuo, Wangmeng, Zhang, David

Understand

This paper presents a Deep convolutional network model for Identity-Aware Transfer (DIAT) of facial attributes.

Given the source input image and the reference attribute, DIAT aims to generate a facial image that owns the reference attribute as well as keeps the same or similar identity to the input image.
In general, our model consists of a mask network and an attribute transform network which work in synergy to generate a photo-realistic facial image with the reference attribute.
Considering that the reference attribute may be only related to some parts of the image, the mask network is introduced to avoid the incorrect editing on attribute irrelevant region.

Built on

S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modeling,” International Journal of Computer Vision , vol. 27, no. 2, pp. 107–126, 1998
1998
Earlier work this paper cites.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing , vol. 13, no. 4, pp. 600–612, April 2004
2004
Earlier work this paper cites.
A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) , vol. 2. IEEE, 2005, pp. 60–65
2005
Earlier work this paper cites.
M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Transactions on Image processing , vol. 15, no. 12, pp. 3736–3745, 2006
2006
Earlier work this paper cites.
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-d transform-domain collaborative filtering,” IEEE Transactions on Image Processing , vol. 16, no. 8, pp. 2080–2095, Aug 2007
2007
Earlier work this paper cites.
N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, “Attribute and simile classifiers for face verification,” in ICCV . IEEE, 2009, pp. 365–372
2009
Earlier work this paper cites.
I. Kemelmacher-Shlizerman, A. Sankar, E. Shechtman, and S. M. Seitz, Being John Malkovich , 2010, pp. 341–353
2010
Earlier work this paper cites.
Y. Fu, G. Guo, and T. S. Huang, “Age synthesis and estimation via faces: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 32, no. 11, pp. 1955–1976, Nov 2010
2010
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in NIPS , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
G. Hinton, N. Srivastava, and K. Swersky, “Rmsprop: Divide the gradient by a running average of its recent magnitude,” Neural networks for machine learning, Coursera lecture 6e , 2012
2012
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114 , 2013
Original
2013
Earlier work this paper cites.
K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034 , 2013
Original
2013
Earlier work this paper cites.
I. Kemelmacher-Shlizerman, S. Suwajanakorn, and S. M. Seitz, “Illumination-aware age progression,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition , June 2014, pp. 3334–3341
2014
Earlier work this paper cites.
J. Gauthier, “Conditional generative adversarial nets for convolutional face generation,” Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter semester , vol. 2014, 2014
2014
Earlier work this paper cites.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS , 2014, pp. 2672–2680
2014
Earlier work this paper cites.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in CVPR , 2014, pp. 580–587
2014
Earlier work this paper cites.
C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in European Conference on Computer Vision . Springer, 2014, pp. 184–199
2014
Earlier work this paper cites.
A. E. Ichim, S. Bouaziz, and M. Pauly, “Dynamic 3d avatar creation from hand-held video input,” ACM Trans. Graph. , vol. 34, no. 4, pp. 45:1–45:14, Jul. 2015
2015
Earlier work this paper cites.
L. Hu, C. Ma, L. Luo, and H. Li, “Single-view hair modeling using a hairstyle database,” ACM Trans. Graph. , vol. 34, no. 4, pp. 125:1–125:9, Jul. 2015
2015
Earlier work this paper cites.
T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. Tenenbaum, “Deep convolutional inverse graphics network,” in NIPS , 2015, pp. 2530–2538
2015
Earlier work this paper cites.

Similar

L. A. Gatys, A. S. Ecker, and M. Bethge, “A neural algorithm of artistic style,” arXiv preprint arXiv:1508.06576 , 2015
Original
2015
Cited alongside, same era.
Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proceedings of International Conference on Computer Vision (ICCV) , 2015
2015
Cited alongside, same era.
C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu, “Deeply-supervised nets.” in AISTATS , vol. 2, no. 3, 2015, p. 6
2015
Cited alongside, same era.
O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference , 2015
2015
Cited alongside, same era.
S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” in ICCV , 2015
2015
Cited alongside, same era.
J.-Y. Zhu, P. Krahenbuhl, E. Shechtman, and A. A. Efros, “Learning a discriminative model for the perception of realism in composite images,” in ICCV , 2015, pp. 3943–3951
2015
Cited alongside, same era.
A. Mahendran and A. Vedaldi, “Understanding deep image representations by inverting them,” in CVPR . IEEE, 2015, pp. 5188–5196
2015
Cited alongside, same era.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2015, pp. 3431–3440
2015
Cited alongside, same era.
A. Dosovitskiy, J. Tobias Springenberg, and T. Brox, “Learning to generate chairs with convolutional neural networks,” in CVPR , 2015, pp. 1538–1546
2015
Cited alongside, same era.
E. L. Denton, S. Chintala, R. Fergus et al. , “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in neural information processing systems , 2015, pp. 1486–1494
2015
Cited alongside, same era.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434 , 2015
Original
2015
Cited alongside, same era.
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2015, pp. 234–241
2015
Cited alongside, same era.

Then

S. Gu, Q. Xie, D. Meng, W. Zuo, X. Feng, and L. Zhang, “Weighted nuclear norm minimization and its applications to low level vision,” International Journal of Computer Vision , pp. 1–26, 2016
2016
Closest in time.
A. van den Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” arXiv preprint arXiv:1601.06759 , 2016
Original
2016
Closest in time.
D. Ulyanov, V. Lebedev, A. Vedaldi, and V. Lempitsky, “Texture networks: Feed-forward synthesis of textures and stylized images,” arXiv preprint arXiv:1603.03417 , 2016
Original
2016
Closest in time.
X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable representation learning by information maximizing generative adversarial nets,” arXiv preprint arXiv:1606.03657 , 2016
Original
2016
Closest in time.
A. Selim, M. Elgharib, and L. Doyle, “Painting style transfer for head portraits using convolutional neural networks,” ACM Transactions on Graphics (TOG) , vol. 35, no. 4, p. 129, 2016
2016
Closest in time.
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 1874–1883
2016
Closest in time.
M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio, “Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1,” arXiv preprint arXiv:1602.02830 , 2016
Original
2016
Closest in time.
A. Garnaev, M. Baykal-Gursoy, and H. V. Poor, “Security games with unknown adversarial strategies,” IEEE transactions on cybernetics , vol. 46, no. 10, pp. 2291–2299, 2016
2016
Closest in time.
F. Zhang, P. P. Chan, B. Biggio, D. S. Yeung, and F. Roli, “Adversarial feature selection against evasion attacks,” IEEE transactions on cybernetics , vol. 46, no. 3, pp. 766–777, 2016
2016
Closest in time.
S. Zhou, T. Xiao, Y. Yang, D. Feng, Q. He, and W. He, “Genegan: Learning object transfiguration and attribute subspace from unpaired data,” arXiv preprint arXiv:1705.04932 , 2017
Original
2017
Closest in time.
W. Shen and R. Liu, “Learning residual images for face attribute manipulation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2017, pp. 1225–1233
2017
Closest in time.
L. Zhang and P. N. Suganthan, “Visual tracking with convolutional random vector functional link network,” IEEE transactions on cybernetics , vol. 47, no. 10, pp. 3243–3253, 2017
2017
Closest in time.
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Transactions on Image Processing , 2017
2017
Closest in time.
K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Transactions on Image Processing , vol. 26, no. 9, pp. 4509–4522, Sept 2017
2017
Closest in time.
B. Du, W. Xiong, J. Wu, L. Zhang, L. Zhang, and D. Tao, “Stacked convolutional denoising auto-encoders for feature representation,” IEEE transactions on cybernetics , vol. 47, no. 4, pp. 1017–1027, 2017
2017
Closest in time.
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in International Conference on Machine Learning , 2017, pp. 214–223
2017
Closest in time.
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved training of wasserstein gans,” arXiv preprint arXiv:1704.00028 , 2017
Original
2017
Closest in time.
R. Yeh, C. Chen, T. Y. Lim, M. Hasegawa-Johnson, and M. N. Do, “Semantic image inpainting with perceptual and contextual losses,” in CVPR , 2017
2017
Closest in time.
J. Huo, Y. Gao, Y. Shi, W. Yang, and H. Yin, “Heterogeneous face recognition by margin-based cross-modality metric learning,” IEEE transactions on cybernetics , vol. 48, no. 6, pp. 1814–1826, 2018
2018
Closest in time.
L. Zheng, S. Duffner, K. Idrissi, C. Garcia, and A. Baskurt, “Pairwise identity verification via linear concentrative metric learning,” IEEE transactions on cybernetics , vol. 48, no. 1, pp. 324–335, 2018
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…