On the bias, risk and consistency of sample means in multi-armed bandits

Understand

The sample mean is among the most well studied estimators in statistics, having many desirable properties such as unbiasedness and consistency.

• However, when analyzing data collected using a multi-armed bandit (MAB) experiment, the sample mean is biased and much remains to be understood about its properties.
• For example, when is it consistent, how large is its bias, and can we bound its mean squared error? This paper delivers a thorough and systematic treatment of the bias, risk and consistency of MAB sample means.
• Specifically, we identify four distinct sources of selection bias (sampling, stopping, choosing and rewinding) and analyze them both separately and together.