This report provides an introduction to the ensmallen numerical optimization library, as well as a deep dive into the technical details of how it works. The library provides a fast and flexible C++ framework for mathematical optimization of arbitrary user-supplied functions. A large set of pre-built optimizers is provided, including many variants of Stochastic Gradient Descent and Quasi-Newton optimizers. Several types of objective functions are supported, including differentiable, separable, constrained, and categorical objective functions. Implementation of a new optimizer requires only one method, while a new objective function requires typically only one or two C++ methods. Through internal use of C++ template metaprogramming, ensmallen provides support for arbitrary user-supplied callbacks and automatic inference of unsupplied methods without any runtime overhead. Empirical comparisons show that ensmallen outperforms other optimization frameworks (such as Julia and SciPy), sometimes by large margins. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.

翻译：本报告介绍了ensmallen数值优化库，并深入探讨了其工作原理的技术细节。该库为任意用户自定义函数的数学优化提供了一个快速且灵活的C++框架。库中预置了大量优化器，包括随机梯度下降和拟牛顿优化器的多种变体。支持多种类型的目标函数，包括可微、可分、带约束和分类目标函数。实现一个新优化器仅需一个方法，而实现一个新目标函数通常只需一两个C++方法。通过内部使用C++模板元编程，ensmallen支持任意用户自定义回调，并能在无运行时开销的情况下自动推断未显式提供的方法。实证对比表明，ensmallen在性能上超越其他优化框架（如Julia和SciPy），有时差距显著。该库可通过https://ensmallen.org获取，并采用宽松的BSD许可协议发布。