Sequence-to-sequence language models can be used to produce abstractive summaries which are coherent, relevant, and concise. Still, model sizes can make deployment in latency-sensitive or web-scale implementations difficult. This paper studies the relationship between model size, structured pruning, inference efficiency, and summarization accuracy on widely used summarization datasets. We show that model accuracy is tied to the encoder size while inference efficiency is connected to the decoder. Using asymmetric pruning can lead to nearly 3x improvement in inference latency with ~1 point loss in Rouge-2. Moreover, we find both the average degradation and the role of asymmetry to be consistent across model sizes and variations in datasets.

翻译：序列到序列语言模型可用于生成连贯、相关且简洁的抽象式摘要。然而，模型规模可能使其在延迟敏感或网络级规模部署中面临困难。本文针对广泛使用的摘要数据集，研究了模型规模、结构化剪枝、推理效率与摘要准确性之间的关系。研究表明，模型准确性与编码器规模密切相关，而推理效率则与解码器相关。采用非对称剪枝可在Rouge-2损失约1个点的情况下，实现近3倍的推理延迟改进。此外，我们还发现平均性能下降幅度以及非对称性的作用在不同模型规模和数据集变化中具有一致性。