Finite-Time Analysis of Kernelised Contextual Bandits

Understand

We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts.

• We focus on the case when the number of actions is too big to sample all of them even once.
• However we assume that we have access to the similarities between actions' contexts and that the expected reward is an arbitrary linear function of the contexts' images in the related reproducing kernel Hilbert space (RKHS).
• We propose KernelUCB, a kernelised UCB algorithm, and give a cumulative regret bound through a frequentist analysis.