Regularization is one of the most fundamental topics in optimization, statistics and machine learning. To get sparsity in estimating a parameter $u\in\mbR^d$, an $\ell_q$ penalty term, $\Vert u\Vert_q$, is usually added to the objective function. What is the probabilistic distribution corresponding to such $\ell_q$ penalty? What is the correct stochastic process corresponding to $\Vert u\Vert_q$ when we model functions $u\in L^q$? This is important for statistically modeling large dimensional objects, e.g. images, with penalty to preserve certainty properties, e.g. edges in the image. In this work, we generalize the $q$-exponential distribution (with density proportional to) $\exp{(- \half|u|^q)}$ to a stochastic process named \emph{$Q$-exponential (Q-EP) process} that corresponds to the $L_q$ regularization of functions. The key step is to specify consistent multivariate $q$-exponential distributions by choosing from a large family of elliptic contour distributions. The work is closely related to Besov process which is usually defined by the expanded series. Q-EP can be regarded as a definition of Besov process with explicit probabilistic formulation and direct control on the correlation length. From the Bayesian perspective, Q-EP provides a flexible prior on functions with sharper penalty ($q<2$) than the commonly used Gaussian process (GP). We compare GP, Besov and Q-EP in modeling functional data, reconstructing images, and solving inverse problems and demonstrate the advantage of our proposed methodology.

翻译：正则化是优化、统计学和机器学习中最基础的主题之一。为了在估计参数$u\in\mbR^d$时获得稀疏性，通常会在目标函数中加入$\ell_q$惩罚项$\Vert u\Vert_q$。与这种$\ell_q$惩罚相对应的概率分布是什么？当我们对$u\in L^q$中的函数进行建模时，与$\Vert u\Vert_q$相对应的正确随机过程是什么？这对于对高维对象（例如图像）进行统计建模至关重要，同时加入惩罚以保持某些确定性属性（例如图像中的边缘）。在本工作中，我们将$q$-指数分布（其密度与$\exp{(- \half|u|^q)}$成正比）推广为一个名为\emph{Q-指数过程（Q-EP）}的随机过程，该过程对应于函数的$L_q$正则化。关键步骤是通过从一大类椭圆轮廓分布中进行选择，来指定一致的多元$q$-指数分布。这项工作与通常由扩展级数定义的Besov过程密切相关。Q-EP可以被视为Besov过程的一种定义，具有显式的概率公式和对相关长度的直接控制。从贝叶斯视角来看，Q-EP为函数提供了一种灵活的先验，其惩罚（$q<2$）比常用的高斯过程（GP）更为尖锐。我们在函数数据建模、图像重建和求解逆问题中比较了高斯过程、Besov过程和Q-EP，并展示了我们提出方法的优势。