Approximation Methods for Partially Observed Markov Decision Processes (POMDPs)

Understand

POMDPs are useful models for systems where the true underlying state is not known completely to an outside observer; the outside observer incompletely knows the true state of the system, and observes a noisy version of the true system state.

• When the number of system states is large in a POMDP that often necessitates the use of approximation methods to obtain near optimal solutions for control.
• This survey is centered around the origins, theory, and approximations of finite-state POMDPs.
• In order to understand POMDPs, it is required to have an understanding of finite-state Markov Decision Processes (MDPs) in \autoref{mdp} and Hidden Markov Models (HMMs) in \autoref{hmm}.