Suppose we are given access to $n$ independent samples from distribution $\mu$ and we wish to output one of them with the goal of making the output distributed as close as possible to a target distribution $\nu$. In this work we show that the optimal total variation distance as a function of $n$ is given by $\tilde\Theta(\frac{D}{f'(n)})$ over the class of all pairs $\nu,\mu$ with a bounded $f$-divergence $D_f(\nu\|\mu)\leq D$. Previously, this question was studied only for the case when the Radon-Nikodym derivative of $\nu$ with respect to $\mu$ is uniformly bounded. We then consider an application in the seemingly very different field of smoothed online learning, where we show that recent results on the minimax regret and the regret of oracle-efficient algorithms still hold even under relaxed constraints on the adversary (to have bounded $f$-divergence, as opposed to bounded Radon-Nikodym derivative). Finally, we also study efficacy of importance sampling for mean estimates uniform over a function class and compare importance sampling with rejection sampling.

翻译：假设我们可以从分布 $\mu$ 中获取 $n$ 个独立样本，并希望输出其中一个样本，使得输出分布尽可能接近目标分布 $\nu$。本文证明，在所有满足有界 $f$-散度 $D_f(\nu\|\mu)\leq D$ 的分布对 $\nu,\mu$ 中，最优总变差距离作为 $n$ 的函数为 $\tilde\Theta(\frac{D}{f'(n)})$。此前，该问题仅在 $\nu$ 关于 $\mu$ 的 Radon-Nikodym 导数一致有界的情况下得到研究。随后，我们考虑一个看似截然不同的平滑在线学习领域的应用，表明即使在对手约束放宽（具有有界 $f$-散度而非有界 Radon-Nikodym 导数）的情况下，关于极小极大遗憾和预言高效算法遗憾的最新结果仍然成立。最后，我们还研究了重要性采样在函数类上均值估计中的有效性，并将其与拒绝采样进行了比较。