Reversible Jump MCMC Simulated Annealing for Neural Networks

Understand

We propose a novel reversible jump Markov chain Monte Carlo (MCMC) simulated annealing algorithm to optimize radial basis function (RBF) networks.

• This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis functions.
• It performs a global search in the joint space of the parameters and number of parameters, thereby surmounting the problem of local minima.
• We also show that by calibrating a Bayesian model, we can obtain the classical AIC, BIC and MDL model selection criteria within a penalized likelihood framework.