Constrained machine learning enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain knowledge into statistical models. Despite its practical importance, no general method exists for the non-convex, non-smooth, stochastic setting that arises naturally in deep learning. We propose the Stochastic Penalty-Barrier Method (SPBM), which extends classical penalty and barrier methods to this setting via exponential dual averaging, a stabilized penalty schedule, and the Moreau envelope to handle non-smoothness. Experiments across multiple settings show that SPBM matches or outperforms existing constrained optimization baselines while incurring only linear runtime overhead compared to unconstrained Adam for up to 10,000 constraints.

翻译：约束机器学习能实现公平性感知训练、物理信息神经网络以及将符号化领域知识整合到统计模型中。尽管其具有实际重要性，但针对深度学习自然产生的非凸、非光滑、随机场景，尚无通用方法。我们提出随机罚障法（SPBM），通过指数对偶平均、稳定化罚项调度及莫罗包络处理非光滑性，将经典罚函数与障碍函数法扩展至该场景。跨多种设置的实验表明，相较于无约束Adam方法，SPBM在多达10,000个约束下仅产生线性运行时开销，同时匹配或超越现有约束优化基线方法。