Critical applications in areas such as medicine, robotics and autonomous systems require compact (i.e., memory efficient), uncertainty-aware neural networks suitable for edge and other resource-constrained deployments. We study compact spectral circulant and block-circulant-with-circulant-blocks (BCCB) layers: FFT-diagonalizable circular convolutions whose weights live directly in the real FFT (RFFT) half (1D) or half-plane (2D). Parameterizing filters in the frequency domain lets us impose simple spectral structure, perform structured variational inference in a low-dimensional weight space, and calculate exact layer spectral norms, enabling inexpensive global Lipschitz bounds and margin-based robustness diagnostics. By placing independent complex Gaussians on the Hermitian support we obtain a discrete instance of the spectral representation of stationary kernels, inducing an exact stationary Gaussian-process prior over filters on the discrete circle/torus. We exploit this to define a practical spectral prior and a Hermitian-aware low-rank-plus-diagonal variational posterior in real coordinates. Empirically, spectral circulant/BCCB layers are effective compact building blocks in both (variational) Bayesian and point estimate regimes: compact Bayesian neural networks on MNIST->Fashion-MNIST, variational heads on frozen CIFAR-10 features, and deterministic ViT projections on CIFAR-10/Tiny ImageNet; spectral layers match strong baselines while using substantially fewer parameters and with tighter Lipschitz certificates.
翻译:在医学、机器人与自主系统等关键应用领域,需要适用于边缘计算及其他资源受限部署场景的紧凑型(即内存高效)且具备不确定性感知能力的神经网络。本研究聚焦紧凑型谱循环层与块循环-循环块(BCCB)层:这类可通过快速傅里叶变换(FFT)对角化的循环卷积层,其权重直接定义于实数FFT(RFFT)半空间(一维)或半平面(二维)。在频域参数化滤波器使我们能够:施加简洁的谱结构约束;在低维权重空间执行结构化变分推断;计算精确的层谱范数,从而实现低成本的全局Lipschitz界估计与基于间隔的鲁棒性诊断。通过在埃尔米特支撑集上设置独立复高斯分布,我们获得了平稳核谱表示的离散实例,从而在离散圆环/环面上诱导出精确的平稳高斯过程先验。基于此,我们构建了实用的谱先验及实数坐标下的埃尔米特感知低秩加对角变分后验。实验表明,谱循环/BCCB层在(变分)贝叶斯与点估计两种范式下均是有效的紧凑构建模块:在MNIST->Fashion-MNIST数据集上的紧凑贝叶斯神经网络、基于冻结CIFAR-10特征的变分头层、以及CIFAR-10/Tiny ImageNet数据集上的确定性ViT投影层中,谱层在显著减少参数量的同时,以更严格的Lipschitz保证达到了强基线模型的性能水平。