Fetching the paper…

Reading the bibliography…

On Random Weights for Texture Generation in One Layer Neural Networks · Around

2016

On Random Weights for Texture Generation in One Layer Neural Networks

Mongia, Mihir, Kumar, Kundan, Erraqabi, Akram et al.

Original
PDF
Post on X

Understand

Recent work in the literature has shown experimentally that one can use the lower layers of a trained convolutional neural network (CNN) to model natural textures.

More interestingly, it has also been experimentally shown that only one layer with random filters can also model textures although with less variability.
In this paper we ask the question as to why one layer CNNs with random filters are so effective in generating textures? We theoretically show that one layer convolutional architectures (without a non-linearity) paired with the an energy function used in previous literature, can in fact preserve and modulate frequency coefficients in a manner so that random weights and pretrained weights will generate the same type of images.
Based on the results of this analysis we question whether similar properties hold in the case where one uses one convolution layer with a non-linearity.

Built on

Hansen, Per Christian, James G. Nagy, and Dianne P. O’leary. Deblurring images: matrices, spectra, and filtering. Vol. 3. Siam, 2006
2006
Earlier work this paper cites.
Glorot, Xavier, and Yoshua Bengio. ”Understanding the difficulty of training deep feedforward neural networks.” In Aistats, vol. 9, pp. 249-256. 2010
2010
Earlier work this paper cites.
Saxe, Andrew, Pang W. Koh, Zhenghao Chen, Maneesh Bhand, Bipin Suresh, and Andrew Y. Ng. ”On random weights and unsupervised feature learning.” In Proceedings of the 28th international conference on machine learning (ICML-11), pp. 1089-1096. 2011
2011
Earlier work this paper cites.

Similar

Galerne, Bruno, Yann Gousseau, and Jean-Michel Morel. ”Random phase textures: Theory and synthesis.” IEEE Transactions on image processing 20, no. 1 (2011): 257-267
2011
Cited alongside, same era.
Horn, Roger A., and Charles R. Johnson. Matrix analysis. Cambridge university press, 2012
Original
2012
Cited alongside, same era.
Texture Synthesis Using Convolutional Neural Networks, 2015;
Original
Leon A. Gatys, Alexander S. Ecker and Matthias Bethge · 2015
Cited alongside, same era.

Then

Champandard, Alex. ”Extreme Style Machines: Using Random Neural Networks To Generate Textures”. nucl.ai. N.p., 2016. Web. 10 Aug. 2016
2016
Closest in time.
A Powerful Generative Model Using Random Weights for the Deep Image Representation, 2016;
Original
Kun He, Yan Wang and John Hopcroft · 2016
Closest in time.
Texture Synthesis Using Shallow Convolutional Networks with Random Filters, 2016;
Original
Ivan Ustyuzhaninov, Wieland Brendel, Leon A. Gatys and Matthias Bethge · 2016
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…