Contextual Markov Decision Processes

Understand

We consider a planning problem where the dynamics and rewards of the environment depend on a hidden static parameter referred to as the context.

• The objective is to learn a strategy that maximizes the accumulated reward across all contexts.
• The new model, called Contextual Markov Decision Process (CMDP), can model a customer's behavior when interacting with a website (the learner).
• The customer's behavior depends on gender, age, location, device, etc.