The l1-regularization is very popular in high dimensional statistics -- it changes a combinatorial problem of choosing which subset of the parameter are zero, into a simple continuous optimization. Using a continuous prior concentrated near zero, the Bayesian counterparts are successful in quantifying the uncertainty in the variable selection problems; nevertheless, the lack of exact zeros makes it difficult for broader problems such as the change-point detection and rank selection. Inspired by the duality of the l1-regularization as a constraint onto an l1-ball, we propose a new prior by projecting a continuous distribution onto the l1-ball. This creates a positive probability on the ball boundary, which contains both continuous elements and exact zeros. Unlike the spike-and-slab prior, this l1-ball projection is continuous and differentiable almost surely, making the posterior estimation amenable to the Hamiltonian Monte Carlo algorithm. We examine the properties, such as the volume change due to the projection, the connection to the combinatorial prior, the minimax concentration rate in the linear problem. We demonstrate the usefulness of exact zeros that simplify the combinatorial problems, such as the change-point detection in time series, the dimension selection of mixture model and the low-rank-plus-sparse change detection in the medical images.
翻译:l1正则化在高维统计中非常流行——它将选择参数子集是否为零的组合问题转化为一个简单的连续优化问题。贝叶斯方法通过采用集中在零附近的连续先验,在变量选择问题的不确定性量化方面取得了成功;然而,缺乏精确零点使得其在更广泛的问题(如变点检测和秩选择)中难以应用。受l1正则化作为l1球约束的二元性启发,我们提出了一种通过将连续分布投影到l1球上的新先验。该方法在球边界上产生了正概率,球边界同时包含连续元素和精确零点。与尖峰-板状先验不同,这种l1球投影几乎处处连续且可微,使得后验估计适用于哈密顿蒙特卡洛算法。我们考察了投影导致的体积变化、与组合先验的联系、线性问题中的极小极大收敛速率等性质。我们展示了精确零点在简化组合问题中的实用性,如时间序列变点检测、混合模型维度选择以及医学图像中的低秩加稀疏变化检测。