Minimum divergence problems under integral constraints appear throughout statistics and probability, including sequential inference, bandit theory, and distributionally robust optimization. In many such settings, dual representations are the key step that convert information-theoretic lower bounds into computationally tractable (and often near-optimal) algorithms. In this paper, we present a general two-stage recipe for deriving dual representations of constrained minimum divergence (in the second argument) for distributions supported on $[0,1]^K$. The first stage derives a dual representation for finitely-supported distributions using classical finite-dimensional convex duality techniques, while the second establishes an abstract interchange argument that lifts this discretized dual to arbitrary distributions. We begin with the simplest case of mean-constrained minimum relative entropy, commonly called $\mathrm{KL}_{\inf}$, and generalize an existing argument from multi-armed bandits literature for $K=1$ to arbitrary dimensions. Our main contribution is to significantly expand the scope of this approach to a broad class of $f$-divergences (beyond relative entropy) and to general integral constraint functionals (beyond the mean constraint). Finally, we illustrate the statistical implications of our results by constructing optimal procedures for sequential testing, estimation, and change detection with observations in $[0,1]^K$.
翻译:积分约束下的最小散度问题广泛出现在统计学和概率论中,包括序贯推断、多臂赌博机理论和分布鲁棒优化。在许多此类场景中,对偶表示是关键步骤,它将信息论下界转化为计算可行(且通常接近最优)的算法。本文提出了一个通用的两阶段方法,用于推导定义在$[0,1]^K$上分布的最小散度(针对第二个参数)在约束条件下的对偶表示。第一阶段利用经典有限维凸对偶技术,推导有限支撑分布的对偶表示;第二阶段通过一个抽象交换论证,将离散化对偶提升至任意分布。我们首先考虑均值约束下最小相对熵的最简情况(通常称为$\mathrm{KL}_{\inf}$),并将多臂赌博机文献中针对$K=1$的现有论证推广到任意维度。本文的主要贡献在于显著扩展了该方法的适用范围,使其涵盖一大类$f-散度$(超越相对熵)和一般积分约束泛函(超越均值约束)。最后,我们通过构建针对$[0,1]^K$上观测值的序贯检验、估计和变点检测的最优程序,展示结果的统计意义。