Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey

Understand

Financial portfolio management is one of the problems that are most frequently encountered in the investment industry.

• Nevertheless, it is not widely recognized that both Kelly Criterion and Risk Parity collapse into Mean Variance under some conditions, which implies that a universal solution to the portfolio optimization problem could potentially exist.
• In fact, the process of sequential computation of optimal component weights that maximize the portfolio's expected return subject to a certain risk budget can be reformulated as a discrete-time Markov Decision Process (MDP) and hence as a stochastic optimal control, where the system being controlled is a portfolio consisting of multiple investment components, and the control is its component weights.
• Consequently, the problem could be solved using model-free Reinforcement Learning (RL) without knowing specific component dynamics.