The algorithmic differentiation (AD) of mathematical functions can be interpreted as a sequence of vertex eliminations in an underlying directed acyclic graph. The problem of determining a minimum-cost elimination ordering, which we call Optimal Vertex Elimination, is NP-complete. Consequently, much effort has been devoted to the design of heuristics. Many of these heuristics are widely believed to perform well in practice, but this hypothesis has so far been difficult to test due to the lack of scalable exact methods. We design and engineer new integer programming formulations for Optimal Vertex Eliminatioin and for a related objective we call Minimum Edge Count. Our implementations scale to graphs one-to-two orders of magnitude larger than existing techniques, enabling the assembly of a corpus of medium-sized graphs for which optimal solutions are known. This corpus facilitates a study of existing heuristics, confirming that on real data popular methods achieve high quality solutions. We also make several theoretical contributions. We give a tight analysis of the forward and reverse modes of AD, and extend our techniques to provide a simple algorithm for Optimal Vertex Elimination with approximation ratio parameterized by the size of a minimum source-sink separator. On the complexity side, we give the first approximation lower bounds for both problems.

翻译：数学函数的算法微分（AD）可解释为有向无环图中顶点消除的序列。确定最小代价消除顺序的问题（称为最优顶点消除）是NP完全的。因此，大量研究致力于启发式算法的设计。尽管许多启发式算法被广泛认为在实践中表现良好，但由于缺乏可扩展的精确方法，这一假设迄今难以验证。我们针对最优顶点消除及其相关目标（称为最小边数）设计并构建了新的整数规划公式。我们的实现可处理比现有技术大一到两个数量级的图，从而构建了一个包含已知最优解的中等规模图语料库。该语料库促进了对现有启发式算法的研究，证实了常用方法在实际数据上能获得高质量解。我们还在理论层面做出贡献：给出了AD前向与反向模式的严格分析，并扩展技术提出了一个参数化于最小源-汇分隔符大小的最优顶点消除近似算法。在复杂性方面，我们首次给出了两个问题的近似下界。