This paper presents a framework for how to incorporate prior sources of information into the design of a sequential experiment. These sources can include previous experiments, expert opinions, or the experimenter's own introspection. We formalize this problem using a Bayesian approach that maps each source to a Bayesian model. These models are aggregated according to their associated posterior probabilities. We evaluate a broad class of policy rules according to three criteria: whether the experimenter learns the parameters of the payoff distributions, the probability that the experimenter chooses the wrong treatment when deciding to stop the experiment, and the average rewards. We show that our framework exhibits several nice finite sample theoretical guarantees, including robustness to any source that is not externally valid.
翻译:本文提出一个框架,用于将先验信息源纳入序贯实验的设计过程。这些信息源可包括既往实验、专家意见或实验者自身的内省分析。我们采用贝叶斯方法对该问题进行形式化描述,将每个信息源映射至一个贝叶斯模型,并根据其对应的后验概率对这些模型进行聚合。依据三项准则评估了一类广泛的策略规则:实验者能否学习收益分布的参数、实验者决定终止实验时选择错误处理的概率,以及平均收益。研究表明,本框架具备多项优异的有限样本理论保证,包括对任何不具备外部效度的信息源具有稳健性。