With the emergence of the Software 3.0 era, there is a growing trend of compressing and integrating large models into software systems, with significant societal implications. Regrettably, in numerous instances, model compression techniques impact the fairness performance of these models and thus the ethical behavior of DNN-powered software. One of the most notable example is the Lottery Ticket Hypothesis (LTH), a prevailing model pruning approach. This paper demonstrates that fairness issue of LTHbased pruning arises from both its subnetwork selection and training procedures, highlighting the inadequacy of existing remedies. To address this, we propose a novel pruning framework, Ballot, which employs a novel conflict-detection-based subnetwork selection to find accurate and fair subnetworks, coupled with a refined training process to attain a high-performance model, thereby improving the fairness of DNN-powered software. By means of this procedure, Ballot improves the fairness of pruning by 38.00%, 33.91%, 17.96%, and 35.82% compared to state-of-the-art baselines, namely Magnitude Pruning, Standard LTH, SafeCompress, and FairScratch respectively, based on our evaluation of five popular datasets and three widely used models. Our code is available at https://anonymous.4open.science/r/Ballot-506E.
翻译:随着软件3.0时代的到来,将大型模型压缩并集成到软件系统中的趋势日益显著,这带来了深远的社会影响。遗憾的是,在许多情况下,模型压缩技术会影响这些模型的公平性表现,进而影响DNN驱动软件的伦理行为。其中最显著的例子是彩票假设(LTH)——一种主流的模型剪枝方法。本文论证了基于LTH的剪枝方法在子网络选择和训练过程中均会引发公平性问题,并指出现有解决方案的不足。为此,我们提出了一种新型剪枝框架Ballot:该框架采用基于冲突检测的新型子网络选择机制来寻找准确且公平的子网络,同时结合精细化的训练流程以获得高性能模型,从而提升DNN驱动软件的公平性。通过此流程,基于我们对五个常用数据集和三个广泛使用模型的评估,Ballot相较于当前最先进的基线方法(分别为幅度剪枝、标准LTH、SafeCompress和FairScratch)将剪枝公平性提升了38.00%、33.91%、17.96%和35.82%。我们的代码公开于https://anonymous.4open.science/r/Ballot-506E。