Invariance describes transformations that do not alter data's underlying semantics. Neural networks that preserve natural invariance capture good inductive biases and achieve superior performance. Hence, modern networks are handcrafted to handle well-known invariances (ex. translations). We propose a framework to learn novel network architectures that capture data-dependent invariances via pruning. Our learned architectures consistently outperform dense neural networks on both vision and tabular datasets in both efficiency and effectiveness. We demonstrate our framework on multiple deep learning models across 3 vision and 40 tabular datasets.

翻译：不变性描述的是不改变数据潜在语义的变换。能够保持自然不变性的神经网络具有更好的归纳偏置，从而获得更优的性能。因此，现代网络通常经过精心设计以处理已知的不变性（例如平移不变性）。我们提出了一个框架，通过剪枝学习能够捕获数据依赖不变性的新型网络架构。在视觉和表格数据集上，我们学习到的架构在效率和有效性上均持续优于稠密神经网络。我们在3个视觉数据集和40个表格数据集上，对多种深度学习模型展示了该框架的有效性。