This thesis presents Regenerative Rejection Sampling (RRS), a novel approximate sampling algorithm inspired by classical Rejection Sampling and Markov Chain Monte Carlo methods. The method constructs a continuous-time regenerative process whose stationary distribution coincides with a target density known only up to a normalizing constant. Unlike standard Rejection Sampling, RRS does not require the existence of a finite constant that upper-bounds the likelihood ratio. As a result, its total variation convergence rate remains exponential for a larger class of scenarios compared to, for example, the Independent Metropolis-Hastings sampler, which requires a finite bounding constant. To explain the workings of the method, we first present a detailed review of renewal and regenerative processes, including their limit theorems, stationary versions, and convergence properties under standard conditions. We explain a coupling proof for exponential convergence of regenerative processes, under the assumption of a spread-out cycle length distribution. We then introduce the RRS algorithm, and derive its convergence rate. Its performance is compared theoretically and empirically with classical MCMC methods. Numerical experiments demonstrate that RRS can exhibit lower autocorrelations and faster effective mixing, both in synthetic examples and in a Bayesian probit regression model applied to a real medical dataset. Moreover, if the algorithm is run until time t, we show that the usual order $O(1/t)$ results for the bias of the time-average estimators, is improved to a bias of $O(1/t^2)$ for the estimator constructed from the RRS method, and provide easy-to-estimate non-asymptotic bounds for this bias.

翻译：本文提出再生拒绝采样（RRS），一种受经典拒绝采样和马尔可夫链蒙特卡洛方法启发的新型近似采样算法。该方法构建了一个连续时间再生过程，其平稳分布与仅已知至归一化常数的目标密度一致。与标准拒绝采样不同，RRS无需要求存在一个有限常数对似然比进行上界限定。因此，与需要有限界限常数的独立梅特罗波利斯-哈斯廷斯采样器相比，其全变差收敛速率在更广泛的场景下仍保持指数级。为阐释该方法的工作原理，我们首先对更新过程和再生过程进行详细综述，包括它们的极限定理、平稳版本及标准条件下的收敛性质。在假定周期长度分布具有扩散性的前提下，我们解释了再生过程指数收敛的耦合证明。随后，我们引入RRS算法并推导其收敛速率。理论上和实验上将其性能与经典MCMC方法进行了比较。数值实验表明，无论是在合成示例还是应用于真实医疗数据集的贝叶斯probit回归模型中，RRS都能展现出更低的自相关性和更快的有效混合。此外，若算法运行至时间t，我们证明，通常时间平均估计量偏差的$O(1/t)$阶数，在RRS方法构建的估计量中改善至$O(1/t^2)$偏差，并为该偏差提供了易于估计的非渐近界限。