In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing. Meanwhile, the structure of neural networks converged back to simpler representations based on piecewise constant and piecewise linear functions such as the Rectified Linear Unit (ReLU), which became the most commonly used type of activation function in neural networks. That made certain types of network structure $\unicode{x2014}$such as the typical fully-connected feedforward neural network$\unicode{x2014}$ amenable to analysis through polyhedral theory and to the application of methodologies such as Linear Programming (LP) and Mixed-Integer Linear Programming (MILP) for a variety of purposes. In this paper, we survey the main topics emerging from this fast-paced area of work, which bring a fresh perspective to understanding neural networks in more detail as well as to applying linear optimization techniques to train, verify, and reduce the size of such networks.

翻译：在过去十年中，深度学习凭借深度神经网络在计算机视觉和自然语言处理等任务中的卓越精度，成为预测建模的主流方法。与此同时，神经网络的结构回归到基于分段常数和分段线性函数（如修正线性单元（ReLU））的更简单表示，这成为神经网络中最常用的激活函数类型。这使得某些网络结构——例如典型的前馈全连接神经网络——可以通过多面体理论进行分析，并应用线性规划（LP）和混合整数线性规划（MILP）等方法实现多种目的。本文综述了这一快速发展领域中出现的主要主题，这些主题为更详细地理解神经网络以及应用线性优化技术训练、验证和简化此类网络提供了全新视角。