This paper focuses on parameter estimation and introduces a new method for lower bounding the Bayesian risk. The method allows for the use of virtually \emph{any} information measure, including R\'enyi's $\alpha$, $\varphi$-Divergences, and Sibson's $\alpha$-Mutual Information. The approach considers divergences as functionals of measures and exploits the duality between spaces of measures and spaces of functions. In particular, we show that one can lower bound the risk with any information measure by upper bounding its dual via Markov's inequality. We are thus able to provide estimator-independent impossibility results thanks to the Data-Processing Inequalities that divergences satisfy. The results are then applied to settings of interest involving both discrete and continuous parameters, including the ``Hide-and-Seek'' problem, and compared to the state-of-the-art techniques. An important observation is that the behaviour of the lower bound in the number of samples is influenced by the choice of the information measure. We leverage this by introducing a new divergence inspired by the ``Hockey-Stick'' Divergence, which is demonstrated empirically to provide the largest lower-bound across all considered settings. If the observations are subject to privatisation, stronger impossibility results can be obtained via Strong Data-Processing Inequalities. The paper also discusses some generalisations and alternative directions.
翻译:本文聚焦于参数估计问题,提出一种基于信息度量的贝叶斯风险下界新方法。该方法可灵活运用任意信息度量,包括Rényi的α散度、φ散度以及Sibson的α互信息。通过将散度视为测度泛函,并利用测度空间与函数空间的对偶性,我们证明:借助马尔可夫不等式对偶上界进行约束,即可用任意信息度量实现风险下界的估计。得益于散度满足的数据处理不等式,我们得以获得独立于估计器的不可能性结论。进一步将方法应用于离散与连续参数场景(包括“Hide-and-Seek”问题),并与现有最优技术进行对比。重要发现是:样本数量对下界行为的影响取决于信息度量的选择。为此我们提出一种基于"Hockey-Stick"散度的新型散度,实验表明其在所有测试场景中均能取得最大下界。当观测数据受隐私保护约束时,通过强数据处理不等式可得到更强的不可能性结论。本文还讨论了若干泛化拓展与替代研究方向。