2018

Correspondence of Deep Neural Networks and the Brain for Visual Textures

Laskar, Md Nasir Uddin, Giraldo, Luis G Sanchez, Schwartz, Odelia

Understand

Deep convolutional neural networks (CNNs) trained on objects and scenes have shown intriguing ability to predict some response properties of visual cortical neurons.

However, the factors and computations that give rise to such ability, and the role of intermediate processing stages in explaining changes that develop across areas of the cortical hierarchy, are poorly understood.
We focused on the sensitivity to textures as a paradigmatic example, since recent neurophysiology experiments provide rich data pointing to texture sensitivity in secondary but not primary visual cortex.
We developed a quantitative approach for selecting a subset of the neural unit population from the CNN that best describes the brain neural recordings.

Built on

B. Julesz, “Experiments in the visual perception of texture,”
1975
Earlier work this paper cites.
H. Tamura, S. Mori, and T. Yamawaki, “Textural features corresponding to visual perception,”
1978
Earlier work this paper cites.
K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,”
1980
Earlier work this paper cites.
B. Julesz, “Textons, the elements of texture perception and their interactions,”
1981
Earlier work this paper cites.
J. R. Bergen and E. H. Adelson, “Early vision and texture perception,”
1988
Earlier work this paper cites.
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Handwritten digit recognition with a back-propagation network,” in
1989
Earlier work this paper cites.
J. Malik and P. Perona, “Preattentive texture discrimination with early vision mechanisms,”
1990
Earlier work this paper cites.
D. J. Heeger, “Normalization of cell responses in cat striate cortex,”
1992
Earlier work this paper cites.
D. J. Heeger, E. P. Simoncelli, and J. A. Movshon, “Computational models of cortical visual processing,”
1996
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”
1998
Earlier work this paper cites.
M. Riesenhuber and T. Poggio, “Hierarchical models of object recognition in cortex,”
1999
Earlier work this paper cites.
W. H. Merigan, “Cortical area V4 is critical for certain texture discrimination, but this effect is not dependent on attention,”
2000
Earlier work this paper cites.
J. Portilla and E. P. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,”
2000
Earlier work this paper cites.
H. Zhou, H. S. Friedman, and R. Von Der Heydt, “Coding of border ownership in monkey visual cortex,”
2000
Earlier work this paper cites.
J. Hegde and D. C. V. Essen, “Selectivity for complex shapes in primate visual area V2,”
2000
Earlier work this paper cites.
A. Hanazawa and H. Komatsu, “Influence of the direction of elemental luminance gradients on the responses of V4 cells to textured surfaces,”
2001
Earlier work this paper cites.
M. Ito and H. Komatsu, “Representation of angles embedded within contour stimuli in area V2 of macaque monkeys,”
2004
Earlier work this paper cites.
L. Zhaoping, “Border ownership from intracortical interactions in visual area V2,”
2005
Earlier work this paper cites.
T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, and T. Poggio, “Robust object recognition with cortex-like mechanisms,”
2007
Earlier work this paper cites.
F. Arcizet, C. Jouffrais, and P. Girard, “Natural textures classification in area V4 of the macaque monkey,”
2008
Earlier work this paper cites.
L. van der Maaten and G. E. Hinton, “Visualizing data using t-SNE,”
2008
Earlier work this paper cites.

Similar

H. Lee, C. Ekanadham, and A. Ng, “Sparse deep belief net model for visual area V2,” in
2008
Cited alongside, same era.
K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, “What is the best multi-stage architecture for object recognition?,” in
2009
Cited alongside, same era.
N. C. Rust and J. J. DiCarlo, “Selectivity and tolerance (”invariance”) both increase as visual information propagates from cortical area V4 to IT,”
2010
Cited alongside, same era.
A. Saxe, P. W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Y. Ng, “On random weights and unsupervised feature learning,” in
2011
Cited alongside, same era.
Y. El-Shamayleh and J. A. Movshon, “Neuronal responses to texture-defined form in macaque visual area V2,”
2011
Cited alongside, same era.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in
2012
Cited alongside, same era.
M. Carandini and D. J. Heeger, “Normalization as a canonical neural computation,”
2012
Cited alongside, same era.
J. Bruna and S. Mallat, “Invariant scattering convolution networks,”
2013
Cited alongside, same era.
L. Sifre and S. Mallat, “Rotation, scaling and deformation invariant scattering for texture discrimination,” in
2013
Cited alongside, same era.
J. Freeman, C. M. Ziemba, D. J. Heeger, E. P. Simoncelli, and J. A. Movshon, “A functional and perceptual signature of the second visual area in primates,”
2013
Cited alongside, same era.
A. S. Nandy, T. O. Sharpee, J. H. Reynolds, and J. F. Mitchell, “The fine structure of shape tuning in area V4,”
2013
Cited alongside, same era.
R. Coen-Cagli and O. Schwartz, “The impact on midlevel vision of statistically optimal divisive normalization in V1,”
2013
Cited alongside, same era.

Then

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,”
2015
Later among the works it cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in
2015
Later among the works it cites.
C. M. Ziemba, R. L. Goris, J. A. Movshon, and E. P. Simoncelli, “Characterizing receptive field selectivity in area v2,” in
2015
Later among the works it cites.
R. Coen-Cagli, A. Kohn, and O. Schwartz, “Flexible gating of contextual modulation during natural vision,”
2015
Later among the works it cites.
H. Hosoya and A. Hyvarinen, “A hierarchical statistical model of natural images. explains tuning properties in V2,”
2015
Later among the works it cites.
D. L. Yamins and J. J. DiCarlo, “Using goal-driven deep learning models to understand sensory cortex,”
2016
Later among the works it cites.
R. M. Cichy, A. Khosla, D. Pantazis, A. Torralba, and A. Oliva, “Deep neural networks predict hierarchical spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence,”
Original
2016
Later among the works it cites.
D. Pospisil, A. Pasupathy, and W. Bair, “Comparing the brain’s representation of shape to that of a deep convolutional neural network,” in
2016
Later among the works it cites.
C. M. Ziemba, J. Freeman, A. Movshon, and E. P. Simoncelli, “Selectivity and tolerance for visual texture in macaque V2,”
2016
Later among the works it cites.
G. Okazawa, S. Tajima, and H. Komatsu, “Gradual development of visual texture-selective properties between macaque areas V2 and V4,”
2016
Later among the works it cites.
PhD thesis, Center for Neural Science, New York University, New York, NY, May 2016
C. M. Ziemba, · 2016
Later among the works it cites.
P. Li, S. Sundar Rangapuram, and M. Slawski, “Methods for Sparse and Low-Rank Recovery under Simplex Constraints,”
2016
Later among the works it cites.
S. A. Cadena, G. H. Denfield, E. Y. Walker, L. A. Gatys, A. S. Tolias, M. Bethge, and A. S. Ecker, “Deep convolutional models improve predictions of macaque V1 responses to natural images,”
2017
Later among the works it cites.
T. S. A. Wallis, C. M. Funke, A. S. Ecker, L. A. Gatys, F. A. Wichmann, and M. Bethge, “A parametric texture model based on deep convolutional features closely matches texture appearance for humans,”
2017
Later among the works it cites.
B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,”
2017
Later among the works it cites.
C. Ziemba, R. Perez, E. Simoncelli, and J. Movshon, “Selectivity of contextual modulation in macaque V1 and V2,” in
2017
Later among the works it cites.
M. N. U. Laskar, L. G. S. Giraldo, and O. Schwartz, “Deep learning captures V2 selectivity for natural textures,” in
2017
Later among the works it cites.
C. Zhuang, Y. Wang, D. Yamins, and X. Hu, “Deep learning predicts correlation between a functional signature of higher visual areas and sparse firing of neurons,”
2017
Later among the works it cites.
[Online; accessed 23-April-2018]
2018
Closest in time.
[Online; accessed 23-April-2018]
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…