We provide a unifying approximate dynamic programming framework that applies to a broad variety of problems involving sequential estimation. We consider first the construction of surrogate cost functions for the purposes of optimization, and we focus on the special case of Bayesian optimization, using the rollout algorithm and some of its variations. We then discuss the more general case of sequential estimation of a random vector using optimal measurement selection, and its application to problems of stochastic and adaptive control. We distinguish between adaptive control of deterministic and stochastic systems: the former are better suited for the use of rollout, while the latter are well suited for the use of rollout with certainty equivalence approximations. As an example of the deterministic case, we discuss sequential decoding problems, and a rollout algorithm for the approximate solution of the Wordle and Mastermind puzzles, recently developed in the paper [BBB22].

翻译：我们首先考虑为优化目的构建代用成本功能,我们侧重于贝叶斯优化的特例,使用推出算法及其某些变异。然后我们讨论使用最佳计量选择对随机矢量进行顺序估算的更一般性案例,以及将其应用于随机矢量控制的问题。我们区分了确定性和随机系统的适应性控制:前者更适合使用推出,而后者则非常适合使用确定性等效近似值的推出。作为确定性案例的一个实例,我们讨论了顺序解码问题,并讨论了最近在论文[BBB22]中开发的关于“Wordle”和“Mastermind”拼图近似解决方案的推出算法。