The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling.

翻译：可微分编程范式是现代科学计算的基础。它指的是计算数值模型输出梯度的数值方法。许多科学模型基于微分方程，其中可微分编程在计算模型灵敏度、反演模型参数以及训练结合微分方程与数据驱动方法的混合模型中发挥着关键作用。此外，认识到反演方法与机器学习之间的强协同效应，为建立适用于这两个领域的统一框架提供了契机。基于微分方程数值解的函数微分并非易事。文献中已提出了基于多种范式的众多方法，每种方法在所研究问题类型上各有利弊。本文对现有计算微分方程数值解导数的技术进行了全面综述。首先，我们讨论了微分方程解梯度在多个科学领域中的重要性。其次，我们阐述了各种方法的数学基础并进行了比较。第三，我们涵盖了计算考量，并探讨了现代科学软件中可用的解决方案。最后但同样重要的是，我们为实践者提供了最佳实践与建议。我们希望这项工作能加速科学模型与数据的融合，并促进科学建模的现代方法发展。