We introduce a novel capacity measure 2sED for statistical models based on the effective dimension. The new quantity provably bounds the generalization error under mild assumptions on the model. Furthermore, simulations on standard data sets and popular model architectures show that 2sED correlates well with the training error. For Markovian models, we show how to efficiently approximate 2sED from below through a layerwise iterative approach, which allows us to tackle deep learning models with a large number of parameters. Simulation results suggest that the approximation is good for different prominent models and data sets.

翻译：我们提出了一种基于有效维度的新型统计模型容量度量——2sED。该新量在模型温和假设下可证明地界定了泛化误差。此外，在标准数据集与主流模型架构上的模拟表明，2sED与训练误差具有良好相关性。针对马尔可夫模型，我们展示了如何通过逐层迭代方法从下方高效近似2sED，这使我们能够处理具有大量参数的深度学习模型。仿真结果表明，该近似方法在不同主流模型与数据集上表现优异。