Is Plug-in Solver Sample-Efficient for Feature-based Reinforcement Learning?

Understand

It is believed that a model-based approach for reinforcement learning (RL) is the key to reduce sample complexity.

• However, the understanding of the sample optimality of model-based RL is still largely missing, even for the linear case.
• This work considers sample complexity of finding an $\epsilon$-optimal policy in a Markov decision process (MDP) that admits a linear additive feature representation, given only access to a generative model.
• We solve this problem via a plug-in solver approach, which builds an empirical model and plans in this empirical model via an arbitrary plug-in solver.