Weathering Ongoing Uncertainty: Learning and Planning in a Time-Varying Partially Observable Environment

Understand

Optimal decision-making presents a significant challenge for autonomous systems operating in uncertain, stochastic and time-varying environments.

• Environmental variability over time can significantly impact the system's optimal decision making strategy for mission completion.
• To model such environments, our work combines the previous notion of Time-Varying Markov Decision Processes (TVMDP) with partial observability and introduces Time-Varying Partially Observable Markov Decision Processes (TV-POMDP).
• We propose a two-pronged approach to accurately estimate and plan within the TV-POMDP: 1) Memory Prioritized State Estimation (MPSE), which leverages weighted memory to provide more accurate time-varying transition estimates; and 2) an MPSE-integrated planning strategy that optimizes long-term rewards while accounting for temporal constraint.