2015

Attribute2Image: Conditional Image Generation from Visual Attributes

Yan, Xinchen, Yang, Jimei, Sohn, Kihyuk et al.

Understand

This paper investigates a novel problem of generating images from visual attributes.

We model the image as a composite of foreground and background and develop a layered generative model with disentangled latent variables that can be learned end-to-end using a variational auto-encoder.
We experiment with natural images of faces and birds and demonstrate that the proposed models are capable of generating realistic and diverse samples with disentangled latent representations.
We use a general energy minimization algorithm for posterior inference of latent variables given novel images.

Built on

Porter, T., Duff, T.: Compositing digital images. In: ACM Siggraph Computer Graphics. vol. 18, pp. 253–259. ACM (1984)
1984
Earlier work this paper cites.
Smolensky, P.: Information processing in dynamical systems: Foundations of harmony theory. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Volume 1 (1986)
1986
Earlier work this paper cites.
Nitzberg, M., Mumford, D.: The 2.1-d sketch. In: ICCV (1990)
1990
Earlier work this paper cites.
Wang, J.Y., Adelson, E.H.: Representing moving images with layers. Image Processing, IEEE Transactions on 3(5), 625–638 (1994)
1994
Earlier work this paper cites.
Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.C.: On advances in statistical modeling of natural images. Journal of mathematical imaging and vision 18(1), 17–33 (2003)
2003
Earlier work this paper cites.
Williams, C.K., Titsias, M.K.: Greedy learning of multiple objects in images using robust statistics and factorial learning. Neural Computation 16(5), 1039–1062 (2004)
2004
Earlier work this paper cites.
Huang, G.B., Ramesh, M., Berg, T.: Labeled faces in the wild: A database for studying. month (07-49) (2007)
2007
Earlier work this paper cites.
Tu, Z.: Learning generative models via discriminative approaches. In: CVPR (2007)
2007
Earlier work this paper cites.
Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: ICCV (2009)
2009
Earlier work this paper cites.
Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML (2009)
2009
Earlier work this paper cites.
Ranzato, M., Mnih, V., Hinton, G.E.: Generating more realistic images using gated mrf’s. In: NIPS (2010)
2010
Earlier work this paper cites.
Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: A matlab-like environment for machine learning. In: BigLearn, NIPS Workshop (2011)
2011
Earlier work this paper cites.
Le Roux, N., Heess, N., Shotton, J., Winn, J.: Learning a generative model of images by factoring appearance and shape. Neural Computation 23(3), 593–650 (2011)
2011
Earlier work this paper cites.
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: ICML (2011)
2011
Earlier work this paper cites.
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD birds-200-2011 dataset (2011)
2011
Earlier work this paper cites.

Similar

Yang, F., Wang, J., Shechtman, E., Bourdev, L., Metaxas, D.: Expression flow for 3D-aware face component transfer. In: SIGGRAPH (2011)
2011
Cited alongside, same era.
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
2012
Cited alongside, same era.
Srivastava, N., Salakhutdinov, R.R.: Multimodal learning with deep Boltzmann machines. In: NIPS (2012)
2012
Cited alongside, same era.
Tang, Y., Salakhutdinov, R., Hinton, G.: Robust boltzmann machines for recognition and denoising. In: CVPR (2012)
2012
Cited alongside, same era.
Yang, Y., Hallman, S., Ramanan, D., Fowlkes, C.C.: Layered object models for image segmentation. PAMI 34(9), 1731–1743 (2012)
2012
Cited alongside, same era.
Bengio, Y., Thibodeau-Laufer, E., Alain, G., Yosinski, J.: Deep generative stochastic networks trainable by backprop. arXiv preprint arXiv:1306.1091 (2013)
Original
2013
Cited alongside, same era.
Isola, P., Liu, C.: Scene collaging: Analysis and synthesis of natural images with semantic layers. In: ICCV (2013)
2013
Cited alongside, same era.
Laput, G.P., Dontcheva, M., Wilensky, G., Chang, W., Agarwala, A., Linder, J., Adar, E.: Pixeltone: a multimodal interface for image editing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2013)
2013
Cited alongside, same era.
Tang, Y., Salakhutdinov, R.: Learning stochastic feedforward neural networks. In: NIPS (2013)
2013
Cited alongside, same era.
Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: NIPS (2014)
2014
Cited alongside, same era.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS. pp. 2672–2680 (2014)
2014
Cited alongside, same era.
Kemelmacher-Shlizerman, I., Suwajanakorn, S., Seitz, S.M.: Illumination-aware age progression. In: CVPR (2014)
2014
Cited alongside, same era.

Then

Reed, S., Sohn, K., Zhang, Y., Lee, H.: Learning to disentangle factors of variation with manifold interaction. In: ICML (2014)
2014
Later among the works it cites.
Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models. In: ICML (2014)
2014
Later among the works it cites.
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Original
2014
Later among the works it cites.
Sohn, K., Shang, W., Lee, H.: Improved multimodal deep learning with variation of information. In: NIPS (2014)
2014
Later among the works it cites.
Zhu, S., Li, C., Loy, C.C., Tang, X.: Transferring landmark annotations for cross-dataset face alignment. arXiv preprint arXiv:1409.0602 (2014)
Original
2014
Later among the works it cites.
Denton, E., Chintala, S., Szlam, A., Fergus, R.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: NIPS (2015)
2015
Closest in time.
Dosovitskiy, A., Springenberg, J.T., Brox, T.: Learning to generate chairs with convolutional neural networks. In: CVPR (2015)
2015
Closest in time.
Gatys, L.A., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: NIPS (2015)
2015
Closest in time.
Gauthier, J.: Conditional generative adversarial nets for convolutional face generation. Tech. rep. (2015)
2015
Closest in time.
Gregor, K., Danihelka, I., Graves, A., Wierstra, D.: DRAW: A recurrent neural network for image generation. In: ICML (2015)
2015
Closest in time.
Kingma, D., Ba, J.: ADAM: A method for stochastic optimization. In: ICLR (2015)
2015
Closest in time.
Kulkarni, T.D., Whitney, W., Kohli, P., Tenenbaum, J.B.: Deep convolutional inverse graphics network. In: NIPS (2015)
2015
Closest in time.
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Original
2015
Closest in time.
Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 (2015)
Original
2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…