Practical Calculation of Gittins Indices for Multi-armed Bandits

Understand

Gittins indices provide an optimal solution to the classical multi-armed bandit problem.

• An obstacle to their use has been the common perception that their computation is very difficult.
• This paper demonstrates an accessible general methodology for the calculating Gittins indices for the multi-armed bandit with a detailed study on the cases of Bernoulli and Gaussian rewards.
• With accompanying easy-to-use open source software, this work removes computation as a barrier to using Gittins indices in these commonly found settings.