Recently, we have witnessed the success of total variation (TV) for many imaging applications. However, traditional TV is defined on the original pixel domain, which limits its potential. In this work, we suggest a new TV regularization defined on the neural domain. Concretely, the discrete data is continuously and implicitly represented by a deep neural network (DNN), and we use the derivatives of DNN outputs w.r.t. input coordinates to capture local correlations of data. As compared with classical TV on the original domain, the proposed TV on the neural domain (termed NeurTV) enjoys two advantages. First, NeurTV is not limited to meshgrid but is suitable for both meshgrid and non-meshgrid data. Second, NeurTV can more exactly capture local correlations across data for any direction and any order of derivatives attributed to the implicit and continuous nature of neural domain. We theoretically reinterpret NeurTV under the variational approximation framework, which allows us to build the connection between classical TV and NeurTV and inspires us to develop variants (e.g., NeurTV with arbitrary resolution and space-variant NeurTV). Extensive numerical experiments with meshgrid data (e.g., color and hyperspectral images) and non-meshgrid data (e.g., point clouds and spatial transcriptomics) showcase the effectiveness of the proposed methods.
翻译:近年来,全变分(TV)正则化在众多成像应用中取得了显著成功。然而,传统TV定义在原始像素域上,这限制了其潜力。本文提出一种定义在神经域上的新型TV正则化方法。具体而言,离散数据通过深度神经网络(DNN)进行连续隐式表示,我们利用DNN输出相对于输入坐标的导数来捕捉数据的局部相关性。与经典原始域TV相比,所提出的神经域TV(称为NeurTV)具有两大优势:首先,NeurTV不受网格结构限制,适用于网格与非网格数据;其次,得益于神经域的隐式连续特性,NeurTV能更精确地捕捉任意方向、任意阶导数所表征的数据局部相关性。我们在变分近似框架下对NeurTV进行理论重构,由此建立经典TV与NeurTV的关联,并启发我们发展出多种变体(例如任意分辨率的NeurTV与空间自适应NeurTV)。通过对网格数据(如彩色图像与高光谱图像)和非网格数据(如点云与空间转录组数据)的大量数值实验,验证了所提方法的有效性。