2011

Kernels for Vector-Valued Functions: a Review

Alvarez, Mauricio A., Rosasco, Lorenzo, Lawrence, Neil D.

Understand

Kernel methods are among the most popular techniques in machine learning.

From a frequentist/discriminative perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces.
From a Bayesian/generative perspective they are the key in the context of Gaussian processes, where the kernel function is also known as the covariance function.
Traditionally, kernel methods have been used in supervised learning problem with scalar outputs and indeed there has been a considerable amount of work devoted to designing and learning kernels.

Built on

Theory of reproducing kernels
N. Aronszajn · 1950
Earlier work this paper cites.
Sous-espaces hilbertiens d’espaces vectoriels topologiques et noyaux associés (noyaux reproduisants)
L. Schwartz · 1964
Earlier work this paper cites.
A correspondence between Bayesian estimation of stochastic processes and smoothing by splines
G. Kimeldorf and G. Wahba · 1970
Earlier work this paper cites.
The intrinsic random functions and their applications
Georges Matheron · 1973
Earlier work this paper cites.
Solutions of Ill Posed Problems
A.N. Tikhonov and V.Y. Arsenin · 1977
Earlier work this paper cites.
Mining Geostatistics
Andre G. Journel and Charles J. Huijbregts · 1978
Earlier work this paper cites.
The analysis of Linear Partial Differential Operators I
Lars Hörmander · 1983
Earlier work this paper cites.
Networks and the best approximation property
F. Girosi and T. Poggio · 1989
Earlier work this paper cites.
Networks for approximation and learning
Tomaso Poggio and Federico Girosi · 1990
Earlier work this paper cites.
Spline Models for Observational Data
Grace Wahba · 1990
Earlier work this paper cites.
Linear coregionalization model: Tools for estimation and choice of cross-variogram matrix
Michel Goulard and Marc Voltz · 1992
Earlier work this paper cites.
Statistics for Spatial Data
Noel A. C. Cressie · 1993
Earlier work this paper cites.
On the pseudo cross-variogram
A. Papritz, H.R. Künsch, and R. Webster · 1993
Earlier work this paper cites.
Universal cokriging under intrinsic coregionalization
Jeffrey D. Helterbrand and Noel Cressie · 1994
Earlier work this paper cites.
Generalized hermite interpolation via matrix-valued conditionally positive definite functions
F.J. Narcowich and J.D. Ward · 1994
Earlier work this paper cites.
Blackbox kriging: spatial prediction without specifying variogram models
Ronald Paul Barry and Jay M. Ver Hoef · 1996
Earlier work this paper cites.
Is learning the n -thing any easier than learning the first?
Sebastian Thrun · 1996
Earlier work this paper cites.
Multitask learning
Rich Caruana · 1997
Earlier work this paper cites.
Geostatistics For Natural Resources Evaluation
Pierre Goovaerts · 1997
Earlier work this paper cites.
Generalized cross-covariances and their estimation
H.R. Künsch, A. Papritz, and F. Bassi · 1997
Earlier work this paper cites.
A process-convolution approach to modeling temperatures in the north atlantic ocean
David M. Higdon · 1998
Earlier work this paper cites.
Non-stationary spatial modeling
David M. Higdon, Jenise Swall, and John Kern · 1998
Earlier work this paper cites.
Statistical learning theory
V. N. Vapnik · 1998
Earlier work this paper cites.
Constructing and fitting models for cokriging and multivariable spatial prediction
Jay M. Ver Hoef and Ronald Paul Barry · 1998
Earlier work this paper cites.
Hierarchical Bayesian space-time models
Christopher K. Wikle, L. Mark Berliner, and Noel Cressie · 1998
Earlier work this paper cites.
Bayesian Classification with Gaussian processes
Christopher K.I. Williams and David Barber · 1998
Earlier work this paper cites.
Vegetation-climate feedbacks in a greenhouse world
Ian Woodward, Mark R. Lomas, and Richard A. Betts · 1998
Earlier work this paper cites.
Learning how to learn is learning with point sets, 1999
Thomas P. Minka and Rosalind W. Picard · 1999
Earlier work this paper cites.
Interpolation of Spatial Data
Michael L. Stein · 1999
Earlier work this paper cites.
Analogies and correspondences between variograms and covariance functions
Tilmann Gneiting, Zoltán Sasvári, and Martin Schlather · 2001
Earlier work this paper cites.
A generalized representer theorem
Bernhard Schšlkopf, Ralf Herbrich, and Alex J. Smola · 2001
Earlier work this paper cites.
On the mathematical foundations of learning
F. Cucker and S. Smale · 2002
Earlier work this paper cites.
Interpolation of nonstationary air pollution processes: a spatial spectral approach
Montserrat Fuentes · 2002
Earlier work this paper cites.
Spectral methods for nonstationary spatial processes
Montserrat Fuentes · 2002
Earlier work this paper cites.
Coregionalization by linear combination of nonorthogonal components
J.A. Vargas Guzmán, A.W. Warrick, and D.E. Myers · 2002
Earlier work this paper cites.
Space and space-time modelling using process convolutions
David M. Higdon · 2002
Earlier work this paper cites.

Similar

Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond
Bernhard Schölkopf and Alexander J. Smola · 2002
Cited alongside, same era.
A kernel-based spectral model for non-Gaussian spatio-temporal processes
Christopher K. Wikle · 2002
Cited alongside, same era.
Fitting a linear model of coregionalization for soil properties using simulated annealing
R.M. Lark and A. Papritz · 2003
Cited alongside, same era.
Fast sparse Gaussian process methods: The informative vector machine
Neil D. Lawrence, Matthias Seeger, and Ralf Herbrich · 2003
Cited alongside, same era.
The Design and Analysis of Computer Experiments
Thomas J. Santner, Brian J. Williams, and William I. Notz · 2003
Cited alongside, same era.
Multi-output support vector regression
E. Vazquez and E. Walter · 2003
Cited alongside, same era.
Multivariate Geostatistics
Hans Wackernagel · 2003
Cited alongside, same era.
Hierarchical Bayesian models for predicting the spread of ecological processes
Christopher K. Wikle · 2003
Cited alongside, same era.
Some properties of regularized kernel methods
E. De Vito, L. Rosasco, A. Caponnetto, M. Piana, and A. Verri · 2004
Cited alongside, same era.
Nonstationary multivariate process modeling through spatially varying coregionalization
Alan E. Gelfand, Alexandra M. Schmidt, Sudipto Banerjee, and C.F. Sirmans · 2004
Cited alongside, same era.
Learning to learn with the informative vector machine
Neil D. Lawrence and John C. Platt · 2004
Cited alongside, same era.
Kernels for multi-task learning
C. A. Micchelli and M. Pontil · 2004
Cited alongside, same era.

Then

Multivariate spatial modeling for geostatistical data using convolved covariance functions
Anandamayee Majumdar and Alan E. Gelfand · 2007
Later among the works it cites.
Gaussian processes for prediction
Michael A. Osborne and Stephen J. Roberts · 2007
Later among the works it cites.
Characterizing the function space for Bayesian kernel models
Natesh S. Pillai, Qiang Wu, Feng Liang, Sayan Mukherjee, and Robert L. Wolpert · 2007
Later among the works it cites.
Multi-task learning for classification with Dirichlet process priors
Ya Xue, Xuejun Liao, and Lawrence Carin · 2007
Later among the works it cites.
Maximum-likelihood estimation for multivariate spatial linear coregionalization models
Hao Zhang · 2007
Later among the works it cites.
Convex multi-task feature learning
Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil · 2008
Later among the works it cites.
An algorithm for transfer learning in a heterogeneous environment
Andreas Argyriou, Andreas Maurer, and Massimiliano Pontil · 2008
Later among the works it cites.
Multi-task Gaussian process prediction
Edwin V. Bonilla, Kian Ming Chai, and Christopher K. I. Williams · 2008
Later among the works it cites.
A dynamic process convolution approach to modeling ambient particulate matter concentrations
Catherine A. Calder · 2008
Later among the works it cites.
Universal kernels for multi-task learning
A. Caponnetto, C.A. Micchelli, M. Pontil, and Y. Ying · 2008
Later among the works it cites.
Gaussian process modelling of latent chemical species: Applications to inferring transcription factor activities
Pei Gao, Antti Honkela, Magnus Rattray, and Neil D. Lawrence · 2008
Later among the works it cites.
Computer model calibration using high dimensional output
Dave Higdon, Jim Gattiker, Brian Williams, and Maria Rightley · 2008
Later among the works it cites.
Clustered multi-task learning: A convex formulation
L. Jacob, F. Bach, and J.P. Vert · 2008
Later among the works it cites.
Clustered multi-task learning: A convex formulation
Laurent Jacob, Francis Bach, and Jean-Philippe Vert · 2008
Later among the works it cites.
Learning divergence-free and curl-free vector fields with matrix-valued kernels
I. Macêdo and R. Castro · 2008
Later among the works it cites.
Calibration and Uncertainty Analysis for Computer Simulations with Multivariate Output
John McFarland, Sankaran Mahadevan, Vicente Romero, and Laura Swiler · 2008
Later among the works it cites.
Towards real-time information processing of sensor network data using computationally efficient multi-output Gaussian processes
Michael A. Osborne, Alex Rogers, Sarvapali D. Ramchurn, Stephen J. Roberts, and Nicholas R. Jennings · 2008
Later among the works it cites.
Gaussian process models for computer experiments with qualitative and quantitative factors
Peter Z. G Qian, Huaiqing Wu, and C. F. Jeff Wu · 2008
Later among the works it cites.
Efficient emulators for multivariate deterministic functions
Jonathan Rougier · 2008
Later among the works it cites.
Graphical multi-task learning
D. Sheldon · 2008
Later among the works it cites.
Gaussian process dynamical models for human motion
Jack M. Wang, David J. Fleet, and Aaron Hertzmann · 2008
Later among the works it cites.
Latent Force Models
Mauricio A. Álvarez, David Luengo, and Neil D. Lawrence · 2009
Later among the works it cites.
Predicting vehicle crashworthiness: Validation of computer models for functional and hierarchical data
M. J. Bayarri, James O. Berger, Marc C. Kennedy, Athanasios Kottas, Rui Paulo, Jerry Sacks, John A. Cafeo, Chin-Hsu Lin, and Jian Tu · 2009
Later among the works it cites.
The Elements of Statistical Learning
Trevor Hastie, Robert Tibshirani, and Jerome Friedman · 2009
Later among the works it cites.
Non-parametric Bayesian kernel models
Feng Liang, Kai Mao, Ming Liao, Sayan Mukherjee, and Mike West · 2009
Later among the works it cites.
Generalization errors and learning curves for regression with multi-task Gaussian processes
Kian Ming Chai · 2010
Later among the works it cites.
Bayesian emulation of complex multi-output and dynamic computer models
Stefano Conti and Anthony O’Hagan · 2010
Later among the works it cites.
Joint covariate selection and joint subspace selection for multiple classification problems
G. Obozinski, B. Taskar, and M.I. Jordan · 2010
Later among the works it cites.
A survey on transfer learning
Sinno Jialin Pan and Qiang Yang · 2010
Later among the works it cites.
Convolved Gaussian Process Priors for Multivariate Regression with Applications to Dynamical Systems
Mauricio A. Álvarez · 2011
Closest in time.
Multi-output learning via spectral filtering
L. Baldassarre, L. Rosasco, A. Barla, and A. Verri · 2011
Closest in time.
Gaussian process preference elicitation
Edwin Bonilla, Shengbo Guo, and Scott Sanner · 2011
Closest in time.
Probabilistic uncertainty analysis of an frf of a structure using a Gaussian process emulator
Thomas E. Fricker, Jeremy E. Oakley, Neil D. Sims, and Keith Worden · 2011
Closest in time.
Bayesian multitask classification with Gaussian process priors
Grigorios Skolidis and Guido Sanguinetti · 2011
Closest in time.
Learning sparse inverse covariance matrices in the presence of confounders
Oliver Stegle, Christoph Lippert, Joris Mooij, Neil Lawrence, and Karsten Borgwardt · 2011
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…