We propose Structured Pruning Adapters (SPAs), a family of compressing, task-switching network adapters, that accelerate and specialize networks using tiny parameter sets. Specifically, we propose a channel- and a block-based SPA and evaluate them with a suite of pruning methods on both computer vision and natural language processing benchmarks. Compared to regular structured pruning with fine-tuning, our channel-SPA improves accuracy by 6.9% on average while using half the parameters at 90% pruned weights. Alternatively, it can learn adaptations with 17x fewer parameters at 70% pruning with 1.6% lower accuracy. Similarly, our block-SPA requires far fewer parameters than pruning with fine-tuning. Our experimental code and Python library of adapters are available at github.com/lukashedegaard/structured-pruning-adapters.

翻译：我们提出结构式节奏适应器(SPAs),这是一组压缩、任务转换网络适配器,可以加速和专门使用微小参数集的网络。具体地说,我们提出一个频道和块状的节能适应器(SPAs),并用一套计算机视觉和自然语言处理基准的修剪方法来评估它们。比起常规结构化微调的修剪,我们的频道-节能适应器平均提高6.9%的精确度,同时使用90%的修剪重量值的一半参数。或者,它可以学习在70%的减速率下用17x的减速参数进行修剪,精确度低1.6%。同样,我们的区节能需要的参数远远少于微调的修剪剪。我们的实验代码和适应器Python图书馆可在 github.com/lukashedegaard/结构-pruning-adopters获得。