This paper focuses on parameter estimation and introduces a new method for lower bounding the Bayesian risk. The method allows for the use of virtually \emph{any} information measure, including R\'enyi's $\alpha$, $\varphi$-Divergences, and Sibson's $\alpha$-Mutual Information. The approach considers divergences as functionals of measures and exploits the duality between spaces of measures and spaces of functions. In particular, we show that one can lower bound the risk with any information measure by upper bounding its dual via Markov's inequality. We are thus able to provide estimator-independent impossibility results thanks to the Data-Processing Inequalities that divergences satisfy. The results are then applied to settings of interest involving both discrete and continuous parameters, including the ``Hide-and-Seek'' problem, and compared to the state-of-the-art techniques. An important observation is that the behaviour of the lower bound in the number of samples is influenced by the choice of the information measure. We leverage this by introducing a new divergence inspired by the ``Hockey-Stick'' Divergence, which is demonstrated empirically to provide the largest lower-bound across all considered settings. If the observations are subject to privatisation, stronger impossibility results can be obtained via Strong Data-Processing Inequalities. The paper also discusses some generalisations and alternative directions.
翻译:本文聚焦于参数估计问题,提出了一种通过信息度量对贝叶斯风险进行下界估计的新方法。该方法可适用于几乎任意信息度量,包括Rényi的α阶散度、φ-散度以及Sibson的α阶互信息。该框架将散度视为测度的泛函,并利用测度空间与函数空间之间的对偶性。特别地,我们证明了通过对偶上界应用马尔可夫不等式,即可利用任意信息度量对风险进行下界约束。基于散度满足的数据处理不等式,我们获得了与估计器无关的不可能性结果。随后将该方法应用于涉及离散与连续参数的典型场景(包括"藏与寻"问题),并与当前最优技术进行比较。关键发现是:下界随样本量的变化行为受信息度量选择的影响。受"Hockey-Stick"散度启发,我们提出一种新型散度,实验证明该散度在所有考察场景中均能给出最大下界。若观测数据经过隐私化处理,可通过强数据处理不等式获得更强的不可能性结果。此外,本文还探讨了该方法的推广形式与替代性研究方向。