Scalable Bilinear $\pi$ Learning Using State and Action Features

Understand

Approximate linear programming (ALP) represents one of the major algorithmic families to solve large-scale Markov decision processes (MDP).

• In this work, we study a primal-dual formulation of the ALP, and develop a scalable, model-free algorithm called bilinear $\pi$ learning for reinforcement learning when a sampling oracle is provided.
• This algorithm enjoys a number of advantages.
• First, it adopts (bi)linear models to represent the high-dimensional value function and state-action distributions, using given state and action features.