A note on strategies for bandit problems with infinitely many arms

A bandit problem consisting of a sequence of n choices (n) from a number of infinitely many Bernoulli arms is considered. The parameters of Bernoulli arms are independent and identically distributed random variables from a common distribution F on the interval [0,1] and F is continuous with F(0)=0 and F(1)=1. The goal is to investigate the asymptotic expected failure rates of k-failure strategies, and obtain a lower bound for the expected failure proportion over all strategies presented in Berry et al. (1997). We show that the asymptotic expected failure rates of k-failure strategies when 0<b1 and a lower bound can be evaluated if the limit of the ratio F(1)–F(t) versus (1–t)^b exists as t1^– for some b>0.