Bayesian Hierarchical Mixtures of Experts

Understand

The Hierarchical Mixture of Experts (HME) is a well-known tree-based model for regression and classification, based on soft probabilistic splits.

• In its original formulation it was trained by maximum likelihood, and is therefore prone to over-fitting.
• Furthermore the maximum likelihood framework offers no natural metric for optimizing the complexity and structure of the tree.
• Previous attempts to provide a Bayesian treatment of the HME model have relied either on ad-hoc local Gaussian approximations or have dealt with related models representing the joint distribution of both input and output variables.