Sequentially Deciding Between Two Experiments

Sequentially Deciding Between Two Experiments for Estimating a Common Success Probability

Janis Hardwick
University of Michigan

Connie Page
Michigan State University

Abstract: To estimate a success probability p, two experiments are available: individual Bernoulli(p) trials or the product of r individual Bernoulli(p) trials. This problem has its roots in reliability where either single components can be tested or a system of r identical components can be tested. A total of N experiments can be performed, and the problem is to sequentially select some combination (allocation) of these two experiments, along with an estimator of p, to achieve low mean square error of the final estimate. This scenario is similar to that of the better-known group testing problem, but here the goal is to estimate failure rates rather than identify defective units. The problem also arises in epidemiological applications such as estimating disease prevalence.

Information maximization considerations, and analysis of the asymptotic mean square error of several estimators, leads to the following adaptive procedure: use the maximum likelihood estimator to estimate p, and if this estimator is below (above) the cut-point a_r, then observe an individual (product) trial at the next stage. An exact analysis of this adaptive procedure for fixed sample sizes shows that it behaves roughly as the asymptotics predict, and that several other estimators and procedures behave far worse than their asymptotics indicate. Further, the adaptive procedure exhibits negative regret over a portion of the parameter range.

Keywords: batch testing, risk assessment, infection rate, grouped data, omniscient allocation, composite testing

Full paper (in compressed Postscript)

Last modified: 5 Mar 1997