Magnitude pruning is one of the mainstream methods in lightweight architecture design whose goal is to extract subnetworks with the largest weight connections. This method is known to be successful, but under very high pruning regimes, it suffers from topological inconsistency which renders the extracted subnetworks disconnected, and this hinders their generalization ability. In this paper, we devise a novel magnitude pruning method that allows extracting subnetworks while guarantying their topological consistency. The latter ensures that only accessible and co-accessible -- impactful -- connections are kept in the resulting lightweight networks. Our solution is based on a novel reparametrization and two supervisory bi-directional networks which implement accessibility/co-accessibility and guarantee that only connected subnetworks will be selected during training. This solution allows enhancing generalization significantly, under very high pruning regimes, as corroborated through extensive experiments, involving graph convolutional networks, on the challenging task of skeleton-based action recognition.

翻译：幅度剪枝是轻量级架构设计的主流方法之一，其目标是通过保留权重连接最大的子网络来提取轻量化模型。该方法虽被证实有效，但在极高剪枝率下存在拓扑不一致性缺陷，导致所提取子网络结构断裂，进而阻碍其泛化能力。本文提出一种新型幅度剪枝方法，可在保证拓扑一致性的前提下提取子网络——这种一致性确保仅保留可访问与共访问（即具有影响力的）连接，从而生成轻量化网络。我们的解决方案基于一种新型重参数化机制及两个监督性双向网络，前者实现可访问性与共访问性，后者确保训练过程中仅选择连通的子网络。通过涉及图卷积网络的骨架动作识别挑战性任务的大量实验验证，该方法在极高剪枝率下能显著提升泛化性能。