What makes waveform-based deep learning so hard? Despite numerous attempts at training convolutional neural networks (convnets) for filterbank design, they often fail to outperform hand-crafted baselines. This is all the more surprising because these baselines are linear time-invariant systems: as such, their transfer functions could be accurately represented by a convnet with a large receptive field. In this article, we elaborate on the statistical properties of simple convnets from the mathematical perspective of random convolutional operators. We find that FIR filterbanks with random Gaussian weights are ill-conditioned for large filters and locally periodic input signals, which both are typical in audio signal processing applications. Furthermore, we observe that expected energy preservation of a random filterbank is not sufficient for numerical stability and derive theoretical bounds for its expected frame bounds.

翻译：是什么让基于波形的深度学习如此困难？尽管人们多次尝试训练用于滤波器组设计的卷积神经网络，但它们往往无法超越手工设计的基线模型。这一现象尤其令人惊讶，因为后者是线性时不变系统：其传递函数本可通过具有大感受野的卷积网络精确表示。本文从随机卷积算子的数学视角出发，深入阐释了简单卷积网络的统计特性。研究发现，在音频信号处理应用中常见的场景——大尺寸滤波器与局部周期输入信号——会导致随机高斯权重的FIR滤波器组病态条件恶化。此外，我们观察到随机滤波器组的期望能量保持不足以确保数值稳定性，并推导了其期望框架界的理论界限。