Bundle adjustment (BA) is a critical technique in various robotic applications such as simultaneous localization and mapping (SLAM), augmented reality (AR), and photogrammetry. BA optimizes parameters such as camera poses and 3D landmarks to align them with observations. With the growing importance of deep learning in perception systems, there is an increasing need to integrate BA with deep learning frameworks for enhanced reliability and performance. However, widely-used C++-based BA libraries, such as GTSAM, g$^2$o, and Ceres Solver, lack native integration with modern deep learning libraries like PyTorch. This limitation affects their flexibility, ease of debugging, and overall implementation efficiency. To address this gap, we introduce an eager-mode BA library seamlessly integrated with PyTorch with high efficiency. Our approach includes a sparsity-aware auto-differentiation design and GPU-accelerated sparse operations designed for 2nd-order optimization. Our eager-mode BA on GPU demonstrates substantial runtime efficiency, achieving an average speedup of 18.5$\times$, 22$\times$, and 23$\times$ across all benchmarks compared to GTSAM, g$^2$o, and Ceres, respectively.

翻译：束调整（BA）是同时定位与地图构建（SLAM）、增强现实（AR）和摄影测量等多种机器人应用中的关键技术。BA通过优化相机位姿和三维地标等参数，使其与观测数据对齐。随着深度学习在感知系统中的重要性日益提升，将BA与深度学习框架集成以增强可靠性和性能的需求不断增加。然而，广泛使用的基于C++的BA库（如GTSAM、g$^2$o和Ceres Solver）缺乏与PyTorch等现代深度学习库的原生集成。这一局限性影响了它们的灵活性、调试便捷性及整体实现效率。为弥补这一不足，我们提出了一种高效集成于PyTorch的急切模式BA库。我们的方法包括专为二阶优化设计的稀疏感知自动微分方案及GPU加速稀疏运算。在GPU上运行的急切模式BA展现出显著的运行时效率，相比GTSAM、g$^2$o和Ceres，在所有基准测试中平均加速分别达到18.5倍、22倍和23倍。