We propose a novel approach to concentration for non-independent random variables. The main idea is to ``pretend'' that the random variables are independent and pay a multiplicative price measuring how far they are from actually being independent. This price is encapsulated in the Hellinger integral between the joint and the product of the marginals, which is then upper bounded leveraging tensorisation properties. Our bounds represent a natural generalisation of concentration inequalities in the presence of dependence: we recover exactly the classical bounds (McDiarmid's inequality) when the random variables are independent. Furthermore, in a ``large deviations'' regime, we obtain the same decay in the probability as for the independent case, even when the random variables display non-trivial dependencies. To show this, we consider a number of applications of interest. First, we provide a bound for Markov chains with finite state space. Then, we consider the Simple Symmetric Random Walk, which is a non-contracting Markov chain, and a non-Markovian setting in which the stochastic process depends on its entire past. To conclude, we propose an application to Markov Chain Monte Carlo methods, where our approach leads to an improved lower bound on the minimum burn-in period required to reach a certain accuracy. In all of these settings, we provide a regime of parameters in which our bound fares better than what the state of the art can provide.
翻译:我们提出了一种针对非独立随机变量的新型集中性方法。核心思想是"假装"这些随机变量相互独立,并为其偏离真实独立性的程度支付一个乘法代价。这一代价通过联合分布与边缘分布乘积之间的Hellinger积分来度量,并借助张量化性质对其上界进行约束。我们的界是依赖存在下集中不等式的自然推广:当随机变量独立时,我们精确恢复了经典界(McDiarmid不等式)。此外,在"大偏差"框架下,即使随机变量呈现非平凡依赖关系,我们仍能获得与独立情况相同的概率衰减速率。为验证这一结论,我们考虑了若干具有实际意义的应用场景。首先,我们给出了有限状态空间马尔可夫链的界;随后研究了非收缩性马尔可夫链——简单对称随机游走,以及随机过程完全依赖其历史轨迹的非马尔可夫设定。最后,我们将其应用于马尔可夫链蒙特卡洛方法,该方法在达到特定精度所需最小燃烧期下界问题上取得了优于现有技术的结果。在所有上述场景中,我们均给出了使所提界优于当前最优方法的参数区间。