Suppose we are given access to $n$ independent samples from distribution $\mu$ and we wish to output one of them with the goal of making the output distributed as close as possible to a target distribution $\nu$. In this work we show that the optimal total variation distance as a function of $n$ is given by $\tilde\Theta(\frac{D}{f'(n)})$ over the class of all pairs $\nu,\mu$ with a bounded $f$-divergence $D_f(\nu\|\mu)\leq D$. Previously, this question was studied only for the case when the Radon-Nikodym derivative of $\nu$ with respect to $\mu$ is uniformly bounded. We then consider an application in the seemingly very different field of smoothed online learning, where we show that recent results on the minimax regret and the regret of oracle-efficient algorithms still hold even under relaxed constraints on the adversary (to have bounded $f$-divergence, as opposed to bounded Radon-Nikodym derivative). Finally, we also study efficacy of importance sampling for mean estimates uniform over a function class and compare importance sampling with rejection sampling.

翻译：假设我们可以获取来自分布 $\mu$ 的 $n$ 个独立样本，并希望输出其中一个样本，以使输出的分布尽可能接近目标分布 $\nu$。本文表明，在所有满足有界 $f$ 散度 $D_f(\nu\|\mu)\leq D$ 的 $\nu, \mu$ 对类别中，作为 $n$ 函数的最优总变差距离由 $\tilde\Theta(\frac{D}{f'(n)})$ 给出。此前，该问题仅在 $\nu$ 关于 $\mu$ 的 Radon-Nikodym 导数一致有界的情况下被研究。随后，我们考虑一个看似截然不同的平滑在线学习领域的应用，在其中展示了即使在敌手约束条件放宽（具有有界 $f$ 散度，而非有界 Radon-Nikodym 导数）的情况下，关于极小极大遗憾和 oracle 高效算法遗憾的最新结果仍然成立。最后，我们还研究了重要性抽样在函数类上的均值估计中的有效性，并将重要性抽样与拒绝抽样进行了比较。