A Review of Feature Selection Methods Based on Mutual Information

Understand

In this work we present a review of the state of the art of information theoretic feature selection methods.

• The concepts of feature relevance, redundance and complementarity (synergy) are clearly defined, as well as Markov blanket.
• The problem of optimal feature selection is defined.
• A unifying theoretical framework is described, which can retrofit successful heuristic criteria, indicating the approximations made by each method.