Automatic differentiation (AD) is a range of algorithms to compute the numeric value of a function's (partial) derivative, where the function is typically given as a computer program or abstract syntax tree. AD has become immensely popular as part of many learning algorithms, notably for neural networks. This paper uses Prolog to systematically derive gradient-based forward- and reverse-mode AD variants from a simple executable specification: evaluation of the symbolic derivative. Along the way we demonstrate that several Prolog features (DCGs, co-routines) contribute to the succinct formulation of the algorithm. We also discuss two applications in probabilistic programming that are enabled by our Prolog algorithms. The first is parameter learning for the Sum-Product Loop Language and the second consists of both parameter learning and variational inference for probabilistic logic programming.

翻译：自动微分（AD）是一类用于计算函数（偏）导数值的算法，其中函数通常以计算机程序或抽象语法树的形式给定。自动微分作为众多学习算法的重要组成部分（尤其是神经网络）已变得极为流行。本文利用Prolog从简单的可执行规范（符号导数的求值）出发，系统推导了基于梯度的前向模式和反向模式自动微分变体。在此过程中，我们证明了Prolog的若干特性（定语句文法、协程）有助于算法的简洁表述。我们还讨论了由Prolog算法所支持的两个概率编程应用：其一是针对求和-乘积循环语言的参数学习，其二是面向概率逻辑编程的参数学习和变分推理。