In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity across diverse applications, including cancer treatment, algorithmic configuration, and chemical process optimization. In this domain, the combination of ML and OR often relies on representing the ML model output using Mixed Integer Programming (MIP) formulations. Numerous studies in the literature have developed such formulations for many ML predictors, with a particular emphasis on Artificial Neural Networks (ANNs) due to their significant interest in many applications. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations that are impractical to solve, thereby impeding scalability. In fact, the ML community has already introduced several techniques to reduce the parameter count of ANNs without compromising their performance, since the substantial size of modern ANNs presents challenges for ML applications as it significantly impacts computational efforts during training and necessitates significant memory resources for storage. In this paper, we showcase the effectiveness of pruning, one of these techniques, when applied to ANNs prior to their integration into MIPs. By pruning the ANN, we achieve significant improvements in the speed of the solution process. We discuss why pruning is more suitable in this context compared to other ML compression techniques, and we identify the most appropriate pruning strategies. To highlight the potential of this approach, we conduct experiments using feed-forward neural networks with multiple layers to construct adversarial examples. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.

翻译：近年来，机器学习（ML）模型与运筹学（OR）工具的融合在众多应用中日益普及，包括癌症治疗、算法配置和化工过程优化。在该领域，ML与OR的结合通常依赖于使用混合整数规划（MIP）公式来表达ML模型的输出。文献中已有大量研究为多种ML预测器（尤其是人工神经网络（ANN））开发了此类公式，因为ANN在许多应用中具有重要价值。然而，ANN通常包含大量参数，导致MIP公式求解困难，从而阻碍了可扩展性。实际上，ML社区已引入多种技术来减少ANN的参数数量而不降低其性能，因为现代ANN的庞大体积对ML应用构成挑战——这不仅显著增加了训练过程中的计算负担，还要求大量的存储资源。本文展示了在将ANN集成到MIP之前，对其实施剪枝（一种参数压缩技术）的有效性。通过剪枝ANN，我们显著提升了求解过程的速度。我们探讨了为何在此背景下剪枝比其他ML压缩技术更具优势，并确定了最合适的剪枝策略。为凸显该方法的潜力，我们使用多层前馈神经网络构建对抗样本进行实验。结果表明，剪枝在显著缩短求解时间的同时不损害最终决策质量，使先前无法求解的实例得以解决。