Janis Hardwick and Quentin F. Stout
This page describes some of our efforts in helping researchers produce adaptive designs that are of use in a range of applications. Towards this end, we emphasize very flexible approaches which can accommodate a variety of cost and risk structures and statistical criteria. Our work applies widely, though we have emphasized its use in clinical trials and have collaborated with researchers in the pharmaceutical industry.
We work in adaptive sampling procedures because of their power and efficiency. Typically, classical ``fixed'' sampling procedures, in which all decisions regarding an experiment are made prior to the observation of data, are suboptimal since learning only occurs at the conclusion of the experiment. Sequential or adaptive procedures, on the other hand, allow adjustments to the design as it is being carried out. In this way, adaptive procedures can make more efficient use of resources without diminishing the statistical power of the procedure. Such designs exploit the inherently sequential nature of many real life processes, and offer great flexibility.
A major disadvantage of adaptive procedures is that sampling distributions of common statistics are affected by the fluidity of the design and generally cannot be described analytically. While there are asymptotic analyses of a few procedures, they rarely give accurate indications of what happens for useful sample sizes. Computer programs overcome this difficulty, and can design and analyze procedures for far more complicated scenarios, ones for which asymptotic results would quite be difficult to obtain. Some problems, however, impose significant computational challenges, and overcoming these has been an important aspect of our work.
By analysis of an experimental procedure we mean determining its properties, such as expected sample length, mean squared error, or number of failures. It can include robustness studies to examine sensitivity to design assumptions and studies to determine operating characteristics. Design means the creation of a procedure, i.e., a rule for deciding what to do at each step of the experiment. Typically there is an objective function, such as the probability that the better treatment will be determined, and the goal of the design is to optimize this. Often we have been able to find the optimal design, and then analyze other designs that have been suggested to determine how close they are to optimal. Some suboptimal designs have desirable properties such as simplicity, but until an optimal design has been found one doesn't know how close to optimal the simpler design is.
Arm will be used to indicate one of the populations that may be sampled, such as treatments in a clinical trial or types of components in a reliability study. This terminology comes from the analogy to the bandit problem, where the arms represent different slot machines and the goal is to optimize profits. To achieve this goal bandits must do some exploration, to collect enough evidence to make informed decisions about which arm is best, while also exploiting the better arm to accummulate profits. This corresponds to design goals such as treating each patient in a study as well as possible. Our work has focused on Bernoulli arms, though it can be extended to more general outcomes.
There are a variety of problems which we work on, especially controlled clinical trials. Important classes include
For several problems we have developed somewhat new goal and design options. These include
Our work relies on extensive development of new algorithms and high-performance programs which implement them. Using these, we have been able to obtain exact evaluations and optimizations for many of the problems listed above. For example, we can analyze 2-arm fully sequential designs with hundreds of observations, and flexible staged designs with multiple stages and more than a hundred observations. By using serial and parallel computers we have been able to solve problems involving fully sequential 3-arm designs, and situations involving 2-arms with delayed responses or missing outcomes. For some of these problems researchers had stated that they were not feasible.
No matter how a design was created, it can be analyzed by either frequentist or Bayesian criteria. We create optimal designs by using a Bayesian approach, partially because it provides a framewok within which dynamic programming can be used for optimization. By using weak priors the design quickly adapts to the data obtained, making it quite robust and giving it desirable frequentist properties as well.
In contrast, trying to directly optimize a frequentist design is often infeasible. For example, in a classical test of hypothesis screening trial with specified error bounds, via a Bayesian approach we create designs with the optimal expected sample size under either the alternative or null hypotheses. These designs use flexible stages and often have sample sizes significantly smaller than those used in practice. For some of these problems, trying to find the optimal sample size by the usual frequentist techniques would take longer than the age of the universe.
Here is a searchable list of our papers and an introduction to the subject of adaptive designs. For further information or consultation help in this area, please contact us:
|Copyright © 2006-2016 Quentin F. Stout.|