Regularization is one of the most fundamental topics in optimization, statistics and machine learning. To get sparsity in estimating a parameter $u\in\mathbb{R}^d$, an $\ell_q$ penalty term, $\Vert u\Vert_q$, is usually added to the objective function. What is the probabilistic distribution corresponding to such $\ell_q$ penalty? What is the correct stochastic process corresponding to $\Vert u\Vert_q$ when we model functions $u\in L^q$? This is important for statistically modeling large dimensional objects, e.g. images, with penalty to preserve certainty properties, e.g. edges in the image. In this work, we generalize the $q$-exponential distribution (with density proportional to) $\exp{(- \frac{1}{2}|u|^q)}$ to a stochastic process named $Q$-exponential (Q-EP) process that corresponds to the $L_q$ regularization of functions. The key step is to specify consistent multivariate $q$-exponential distributions by choosing from a large family of elliptic contour distributions. The work is closely related to Besov process which is usually defined by the expanded series. Q-EP can be regarded as a definition of Besov process with explicit probabilistic formulation and direct control on the correlation length. From the Bayesian perspective, Q-EP provides a flexible prior on functions with sharper penalty ($q<2$) than the commonly used Gaussian process (GP). We compare GP, Besov and Q-EP in modeling functional data, reconstructing images, and solving inverse problems and demonstrate the advantage of our proposed methodology.

翻译：正则化是优化、统计学和机器学习中最基本的主题之一。为了在估计参数 \(u \in \mathbb{R}^d\) 时获得稀疏性，通常会在目标函数中添加 \(\ell_q\) 惩罚项 \(\|u\|_q\)。对应于这种 \(\ell_q\) 惩罚的概率分布是什么？当我们对函数 \(u \in L^q\) 进行建模时，与 \(\|u\|_q\) 对应的正确随机过程又是什么？这对于统计建模高维对象（例如图像）具有重要意义，通过惩罚项来保持诸如图像边缘等确定性属性。在本工作中，我们将 \(q\)-指数分布（其密度正比于 \(\exp(-\frac{1}{2}|u|^q)\)）推广为一个名为 \(Q\)-指数过程（Q-EP）的随机过程，该过程对应于函数的 \(L_q\) 正则化。关键步骤是通过从一大类椭圆等高线分布中选择，来指定一致的多元 \(q\)-指数分布。这项工作与通常通过级数展开定义的贝索夫过程密切相关。Q-EP可视为贝索夫过程的一种定义，具有显式的概率公式和对相关长度的直接控制。从贝叶斯视角来看，Q-EP为函数提供了一种灵活的先验，其惩罚项（\(q<2\)）比常用的高斯过程（GP）更为尖锐。我们比较了GP、贝索夫过程和Q-EP在函数型数据建模、图像重建和反问题求解中的表现，并展示了我们提出方法的优势。