Training deep neural networks with noise and data heterogeneity is a major challenge. We introduce Lightweight Learnable Adaptive Weighting (LiLAW), a method that dynamically adjusts the loss weight of each training sample based on its evolving difficulty, categorized as easy, moderate, and hard, using only three global learnable scalar parameters. LiLAW learns to adaptively prioritize samples by updating these parameters with a single gradient descent step on a validation mini-batch after each training mini-batch, without requiring a clean, unbiased validation set. Experiments across general and medical imaging datasets, several noise types and levels, loss functions, and architectures with and without pretraining, including linear probing and full fine-tuning, show that LiLAW consistently improves accuracy and AUROC, especially in higher-noise settings, without requiring excessive tuning. We also obtain state-of-the-art results incorporating synthetic and augmented data from SynPAIN, GAITGen, ECG5000, and improved fairness on the Adult dataset. LiLAW is lightweight, practical, and computationally efficient, making it an effective, scalable approach to boost generalization and robustness across diverse deep learning training setups, especially in resource-constrained settings.

翻译：摘要：在存在噪声和数据异质性的条件下训练深度神经网络是一项重大挑战。我们提出轻量级可学习自适应加权（LiLAW）方法，该方法仅使用三个全局可学习标量参数，根据每个训练样本动态变化的难度（分为简单、中等和困难三类）自适应调整其损失权重。LiLAW通过在每批训练样本后对验证小批量执行单步梯度下降来更新这些参数，从而学习自适应地优先处理样本，而无需干净无偏的验证集。在通用和医学影像数据集、多种噪声类型与水平、不同损失函数及有无预训练（包括线性探测和全微调）的架构上的实验表明，LiLAW始终能提升准确率和AUROC，尤其在高噪声设置下效果显著，且无需过度调参。我们还通过整合来自SynPAIN、GAITGen、ECG5000的合成与增强数据获得了最先进的结果，并在Adult数据集上改善了公平性。LiLAW轻量、实用且计算高效，是一种有效且可扩展的方法，可在各种深度学习训练设置（尤其是资源受限环境）中提升泛化能力和鲁棒性。