Traditional signal processing methods relying on mathematical data generation models have been cast aside in favour of deep neural networks, which require vast amounts of data. Since the theoretical sample complexity is nearly impossible to evaluate, these amounts of examples are usually estimated with crude rules of thumb. However, these rules only suggest when the networks should work, but do not relate to the traditional methods. In particular, an interesting question is: how much data is required for neural networks to be on par or outperform, if possible, the traditional model-based methods? In this work, we empirically investigate this question in two simple examples, where the data is generated according to precisely defined mathematical models, and where well-understood optimal or state-of-the-art mathematical data-agnostic solutions are known. A first problem is deconvolving one-dimensional Gaussian signals and a second one is estimating a circle's radius and location in random grayscale images of disks. By training various networks, either naive custom designed or well-established ones, with various amounts of training data, we find that networks require tens of thousands of examples in comparison to the traditional methods, whether the networks are trained from scratch or even with transfer-learning or finetuning.

翻译：依赖数学数据生成模型的传统信号处理方法已被需要海量数据的深度神经网络所取代。由于理论样本复杂度几乎无法评估，这些样本数量通常通过粗糙的经验法则来估算。然而，这些规则仅能指出网络何时有效，却未能与传统方法建立关联。特别值得注意的是一个有趣的问题：为了与基于模型的传统方法持平甚至超越其性能（如果可能），神经网络需要多少数据？在本工作中，我们通过两个简单示例对这一课题进行实证研究：其一涉及基于精确定义数学模型生成的数据，其二则对应已知最优或最先进的、与数据无关的数学解决方案。第一个问题是反卷积一维高斯信号，第二个问题是从随机灰度圆盘图像中估计圆的半径和位置。通过训练各类网络（包括简单自定义网络与成熟网络）并采用不同规模的训练数据，我们发现无论是从零开始训练，还是采用迁移学习或微调方式，与传统方法相比，神经网络都需要数万个样本。