The deep neural network (DNN) has been proven effective in various domains. However, they often struggle to perform well on certain minority groups during inference, despite showing strong performance on the majority of data groups. This is because over-parameterized models learned \textit{bias attributes} from a large number of \textit{bias-aligned} training samples. These bias attributes are strongly spuriously correlated with the target variable, causing the models to be biased towards spurious correlations (i.e., \textit{bias-conflicting}). To tackle this issue, we propose a novel \textbf{re}weighted \textbf{s}parse \textbf{t}raining framework, dubbed as \textit{\textbf{REST}}, which aims to enhance the performance of biased data while improving computation and memory efficiency. Our proposed REST framework has been experimentally validated on three datasets, demonstrating its effectiveness in exploring unbiased subnetworks. We found that REST reduces the reliance on spuriously correlated features, leading to better performance across a wider range of data groups with fewer training and inference resources. We highlight that the \textit{REST} framework represents a promising approach for improving the performance of DNNs on biased data, while simultaneously improving computation and memory efficiency. By reducing the reliance on spurious correlations, REST has the potential to enhance the robustness of DNNs and improve their generalization capabilities. Code is released at \url{https://github.com/zhao1402072392/REST}

翻译：深度神经网络已在多个领域证实其有效性。然而，尽管在多数数据组上表现优异，它们在推理时往往难以在特定少数群体上取得良好性能。这是因为过参数化模型从大量“偏差对齐”训练样本中学习了“偏差属性”。这些偏差属性与目标变量存在强烈的虚假关联，导致模型倾向于利用虚假相关性（即“偏差冲突”）。为解决此问题，我们提出了一种新颖的**重**加权**稀**疏**训**练框架，命名为**REST**，旨在提升偏差数据性能的同时提高计算与内存效率。我们的REST框架已在三个数据集上通过实验验证，证明了其在探索无偏差子网络方面的有效性。我们发现，REST减少了对虚假相关特征的依赖，从而以更少的训练和推理资源实现了更广数据组上的更优性能。我们强调，REST框架是一种有望改善深度神经网络在偏差数据上性能的前沿方法，同时能提高计算与内存效率。通过降低对虚假相关性的依赖，REST具有增强深度神经网络鲁棒性并提升其泛化能力的潜力。代码发布在：\url{https://github.com/zhao1402072392/REST}