Prior knowledge and symbolic rules in machine learning are often expressed in the form of label constraints, especially in structured prediction problems. In this work, we compare two common strategies for encoding label constraints in a machine learning pipeline, regularization with constraints and constrained inference, by quantifying their impact on model performance. For regularization, we show that it narrows the generalization gap by precluding models that are inconsistent with the constraints. However, its preference for small violations introduces a bias toward a suboptimal model. For constrained inference, we show that it reduces the population risk by correcting a model's violation, and hence turns the violation into an advantage. Given these differences, we further explore the use of two approaches together and propose conditions for constrained inference to compensate for the bias introduced by regularization, aiming to improve both the model complexity and optimal risk.

翻译：在机器学习中，先验知识与符号规则常以标签约束的形式表达，尤其是在结构化预测问题中。本研究通过量化其对模型性能的影响，比较了在机器学习流程中编码标签约束的两种常见策略：基于约束的正则化与约束化推理。针对正则化，我们证明其通过排除与约束不一致的模型，收窄了泛化差距；然而，其对小幅违规的偏好会引入偏向次优模型的偏差。针对约束化推理，我们证明其通过修正模型的违规行为降低了总体风险，从而将违规转化为优势。基于这些差异，我们进一步探讨了两种方法的联合应用，并提出约束化推理应补偿正则化所引入偏差的条件，旨在同时提升模型复杂度与最优风险。