An underlying mechanism for successful deep learning (DL) with a limited deep architecture and dataset, namely VGG-16 on CIFAR-10, was recently presented based on a quantitative method to measure the quality of a single filter in each layer. In this method, each filter identifies small clusters of possible output labels, with additional noise selected as labels out of the clusters. This feature is progressively sharpened with the layers, resulting in an enhanced signal-to-noise ratio (SNR) and higher accuracy. In this study, the suggested universal mechanism is verified for VGG-16 and EfficientNet-B0 trained on the CIFAR-100 and ImageNet datasets with the following main results. First, the accuracy progressively increases with the layers, whereas the noise per filter typically progressively decreases. Second, for a given deep architecture, the maximal error rate increases approximately linearly with the number of output labels. Third, the average filter cluster size and the number of clusters per filter at the last convolutional layer adjacent to the output layer are almost independent of the number of dataset labels in the range [3, 1,000], while a high SNR is preserved. The presented DL mechanism suggests several techniques, such as applying filter's cluster connections (AFCC), to improve the computational complexity and accuracy of deep architectures and furthermore pinpoints the simplification of pre-existing structures while maintaining their accuracies.

翻译：针对有限深度架构与数据集（如CIFAR-10上的VGG-16）下成功深度学习的底层机制，近期基于量化单层滤波器质量的度量方法被提出。该方法中，每个滤波器识别可能的输出标签的小簇，并选择簇外标签作为附加噪声。该特征随网络层级递增逐步锐化，从而提升信噪比与精度。本研究在CIFAR-100与ImageNet数据集上训练的VGG-16与EfficientNet-B0模型中验证了所提出的普适性机制，主要结论如下：第一，精度随层级递增而渐进提升，而单层噪声典型递减；第二，对于给定深度架构，最大错误率近似随输出标签数量线性增加；第三，靠近输出层的最后一层卷积层中，平均滤波器簇尺寸及每滤波器簇数量在[3, 1,000]标签范围内几乎独立于数据集标签数，且保持高信噪比。所提出的深度学习机制衍生出若干技术（如滤波器簇连接应用），可改善深度架构的计算复杂度与精度，同时指出在保持精度前提下简化现有结构的可能性。