Convolution is a fundamental operation in image processing and machine learning. Aimed primarily at maintaining image size, padding is a key ingredient of convolution, which, however, can introduce undesirable boundary effects. We present a non-padding-based method for size-keeping convolution based on the preservation of differential characteristics of kernels. The main idea is to make convolution over an incomplete sliding window "collapse" to a linear differential operator evaluated locally at its central pixel, which no longer requires information from the neighbouring missing pixels. While the underlying theory is rigorous, our final formula turns out to be simple: the convolution over an incomplete window is achieved by convolving its nearest complete window with a transformed kernel. This formula is computationally lightweight, involving neither interpolation or extrapolation nor restrictions on image and kernel sizes. Our method favours data with smooth boundaries, such as high-resolution images and fields from physics. Our experiments include: i) filtering analytical and non-analytical fields from computational physics and, ii) training convolutional neural networks (CNNs) for the tasks of image classification, semantic segmentation and super-resolution reconstruction. In all these experiments, our method has exhibited visible superiority over the compared ones.
翻译:卷积是图像处理和机器学习中的基础运算。为保持图像尺寸而引入的填充是卷积的关键组成部分,但这一操作可能产生有害的边界效应。本文提出一种基于核微分特性保持的无填充尺寸保持卷积方法。其核心思想是使不完整滑动窗口上的卷积"坍缩"为在其中心像素处局部评估的线性微分算子,从而无需来自相邻缺失像素的信息。尽管底层理论具有严格性,但最终公式却简洁明了:通过将最近完整窗口与变换核进行卷积来实现不完整窗口上的卷积。该公式计算轻量,无需插值或外推,亦不对图像和核尺寸施加限制。本方法尤其适用于具有平滑边界的数据,如高分辨率图像和物理场。实验包括:i) 计算物理学中解析与非解析场的滤波,ii) 训练用于图像分类、语义分割和超分辨率重建任务的卷积神经网络(CNN)。所有实验均表明,本方法相较于对比方法具有显著优势。