A New Computation-driven Framework for Adaptive Experimentation

Hongseok Namkoong, PhD

Assistant Professor in the Decision, Risk, and Operations division

Columbia Business School

4/21/2023

11 am – 12pm CST

WebEx: https://utsa.webex.com/utsa/j.php?MTID=m568fe3d73989cb26d676f896e952c3c0

Abstract:  Experimentation serves as the foundation of scientific decision-making. Adaptive allocation of measurement effort can significantly improve statistical power. However, implementing standard bandit algorithms, which assume continual reallocation of measurement effort, is challenging due to delayed feedback and infrastructural or organizational difficulties. To address this, we introduce a new framework for adaptive experimentation, motivated by practical instances involving a limited number of reallocation epochs in which outcomes are measured in batches. Our framework diverges from the traditional theory-driven paradigm by utilizing computational tools for algorithmic design.

We observe that normal approximations, which are universal in statistical inference, can also guide the design of scalable adaptive designs. By deriving an asymptotic sequential experiment, we formulate a dynamic program that can leverage prior information on average rewards. We propose a simple iterative planning method called Residual Horizon Optimization, which selects sampling allocations by optimizing a planning objective. Our method significantly improves statistical power over standard adaptive policies, even when compared to Bayesian bandit algorithms (e.g., Thompson sampling) that require full distributional knowledge of individual rewards. Overall, we expand the scope of adaptive experimentation to settings that pose difficulties for standard adaptive policies, including problems with a small number of reallocation epochs, low signal-to-noise ratio, and unknown reward distributions.

This work was led by Ethan Che. Paper link: https://arxiv.org/abs/2303.11582.