2019

Kernel computations from large-scale random features obtained by Optical Processing Units

Ohana, Ruben, Wacker, Jonas, Dong, Jonathan et al.

Understand

Approximating kernel functions with random features (RFs)has been a successful application of random projections for nonparametric estimation.

However, performing random projections presents computational challenges for large-scale problems.
Recently, a new optical hardware called Optical Processing Unit (OPU) has been developed for fast and energy-efficient computation of large-scale RFs in the analog domain.
More specifically, the OPU performs the multiplication of input vectors by a large random matrix with complex-valued i.i.d.

“Sparse greedy matrix approximation for machine learning,”
Alex J Smola and Bernhard Schölkopf, · 2000
Earlier work this paper cites.
Learning with kernels: support vector machines, regularization, optimization, and beyond
Bernhard Schölkopf, Alexander J Smola, Francis Bach, et al., · 2002
Earlier work this paper cites.
“Optimal rates for the regularized least-squares algorithm,”
Andrea Caponnetto and Ernesto De Vito, · 2007
Earlier work this paper cites.
“Random features for large-scale kernel machines,”
Ali Rahimi and Benjamin Recht, · 2008
Earlier work this paper cites.
“Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning,”
Ali Rahimi and Benjamin Recht, · 2009
Earlier work this paper cites.
“Imagenet classification with deep convolutional neural networks,”
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, · 2012
Earlier work this paper cites.
“Divide and conquer kernel ridge regression,”
Yuchen Zhang, John Duchi, and Martin Wainwright, · 2013
Earlier work this paper cites.

“Fastfood-approximating kernel expansions in loglinear time,”
Quoc Le, Tamás Sarlós, and Alex Smola, · 2013
Cited alongside, same era.
“Imaging with nature: Compressive imaging using a multiply scattering medium,”
Antoine Liutkus, David Martina, Sébastien Popoff, Gilles Chardon, Ori Katz, Geoffroy Lerosey, Sylvain Gigan, Laurent Daudet, and Igor Carron, · 2014
Cited alongside, same era.
“Very deep convolutional networks for large-scale image recognition,”
Original
K. Simonyan and A. Zisserman, · 2014
Cited alongside, same era.
“Less is more: Nyström computational regularization,”
Alessandro Rudi, Raffaello Camoriano, and Lorenzo Rosasco, · 2015
Cited alongside, same era.
“Deep residual learning for image recognition,”
Original
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, · 2015
Cited alongside, same era.
“ImageNet Large Scale Visual Recognition Challenge,”
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei, · 2015
Cited alongside, same era.

“Random projections through multiple optical scattering: Approximating kernels at the speed of light,”
A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, · 2016
Later among the works it cites.
“Falkon: An optimal large scale kernel method,”
Alessandro Rudi, Luigi Carratino, and Lorenzo Rosasco, · 2017
Later among the works it cites.
“Random feature expansions for deep Gaussian processes,”
Kurt Cutajar, Edwin Bonilla, Pietro Michiardi, and Maurizio Filippone, · 2017
Later among the works it cites.
“Scaling up echo-state networks with multiple light scattering,”
Jonathan Dong, Sylvain Gigan, Florent Krzakala, and Gilles Wainrib, · 2018
Later among the works it cites.
“Newma: a new method for scalable model-free online change-point detection,”
Original
Nicolas Keriven, Damien Garreau, and Iacopo Poli, · 2018
Later among the works it cites.
“Optical reservoir computing using multiple light scattering for chaotic systems prediction,”
Jonathan Dong, Mushegh Rafayelyan, Florent Krzakala, and Sylvain Gigan, · 2019
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…