Deep Neural Networks are powerful tools for solving machine learning problems, but their training often involves dense and costly parameter updates. In this work, we use a novel Max-Plus neural architecture in which classical addition and multiplication are replaced with maximum and summation operations respectively. This is a promising architecture in terms of interpretability, but its training is challenging. A particular feature is that this algebraic structure naturally induces sparsity in the subgradients, as only neurons that contribute to the maximum affect the loss. However, standard backpropagation fails to exploit this sparsity, leading to unnecessary computations. In this work, we focus on the minimization of the worst sample loss which transfers this sparsity to the optimization loss. To address this, we propose a sparse subgradient algorithm that explicitly exploits the algebraic sparsity. By tailoring the optimization procedure to the non-smooth nature of Max-Plus models, our method achieves more efficient updates while retaining theoretical guarantees. This highlights a principled path toward bridging algebraic structure and scalable learning.

翻译：深度神经网络是解决机器学习问题的强大工具，但其训练过程通常涉及密集且计算代价高昂的参数更新。在本研究中，我们采用一种新颖的Max-Plus神经架构，其中经典的加法与乘法运算分别被最大值运算与求和运算所替代。该架构在可解释性方面具有显著优势，但其训练过程颇具挑战性。该代数结构的一个典型特征是：由于仅对最大值产生贡献的神经元会影响损失函数，因此次梯度天然具有稀疏性。然而，标准反向传播算法未能利用这种稀疏性，导致大量不必要的计算。本研究聚焦于最小化最差样本损失，该优化目标将这种稀疏性传递至优化损失函数。为此，我们提出一种显式利用代数稀疏性的稀疏次梯度算法。通过针对Max-Plus模型的非光滑特性定制优化流程，我们的方法在保持理论保证的同时实现了更高效的参数更新。这为连接代数结构与可扩展学习提供了一条基于原理的可行路径。