Suppose we are given access to $n$ independent samples from distribution $\mu$ and we wish to output one of them with the goal of making the output distributed as close as possible to a target distribution $\nu$. In this work we show that the optimal total variation distance as a function of $n$ is given by $\tilde\Theta(\frac{D}{f'(n)})$ over the class of all pairs $\nu,\mu$ with a bounded $f$-divergence $D_f(\nu\|\mu)\leq D$. Previously, this question was studied only for the case when the Radon-Nikodym derivative of $\nu$ with respect to $\mu$ is uniformly bounded. We then consider an application in the seemingly very different field of smoothed online learning, where we show that recent results on the minimax regret and the regret of oracle-efficient algorithms still hold even under relaxed constraints on the adversary (to have bounded $f$-divergence, as opposed to bounded Radon-Nikodym derivative). Finally, we also study efficacy of importance sampling for mean estimates uniform over a function class and compare importance sampling with rejection sampling.

翻译：假设我们可以从分布 $\mu$ 中获取 $n$ 个独立样本，并希望输出其中一个样本，目标是使输出尽可能接近目标分布 $\nu$。本文证明，在所有满足有界 $f$ 散度 $D_f(\nu\|\mu)\leq D$ 的分布对 $\nu,\mu$ 类上，最优总变差距离作为 $n$ 的函数为 $\tilde\Theta(\frac{D}{f'(n)})$。此前，该问题仅针对 $\nu$ 关于 $\mu$ 的 Radon-Nikodym 导数一致有界的情形进行了研究。随后，我们考虑该结果在看似截然不同的平滑在线学习领域中的应用，表明即使在对手约束放宽（具有有界 $f$ 散度，而非有界 Radon-Nikodym 导数）的条件下，关于极小极大遗憾和预言机高效算法遗憾的最新结果仍然成立。最后，我们还研究了重要性抽样在函数类均值估计中的有效性，并将重要性抽样与拒绝抽样进行了比较。