The algorithms are quite general, and give users flexibility in incorporating practical considerations in the design of experiments. To show the usability of these algorithms, they are applied to a bandit problem and an estimation problem. It is observed that the expected number of switches grows approximately as the square root of the sample size, for sample sizes up to a few hundred. It is also observed that one can dramatically reduce the number of switches without substantially affecting the expected value of the objective function. Thus one need sacrifice only a small amount of statistical objective in order to achieve significant gains in practicality.

Keywords: adaptive sampling, switching costs, constraints, bandit, estimation, dynamic programming, optimal tradeoffs