Nonlinear extensions of the Kalman filter (KF), such as the extended Kalman filter (EKF) and the unscented Kalman filter (UKF), are indispensable for state estimation in complex dynamical systems, yet the conditions for a nonlinear KF to provide robust and accurate estimations remain poorly understood. This work proposes a theoretical framework that identifies the causes of failure and success in certain nonlinear KFs and establishes guidelines for their improvement. Central to our framework is the concept of covariance compensation: the deviation between the covariance predicted by a nonlinear KF and that of the EKF. With this definition and detailed theoretical analysis, we derive three design guidelines for nonlinear KFs: (i) invariance under orthogonal transformations, (ii) sufficient covariance compensation beyond the EKF baseline, and (iii) selection of compensation magnitude that favors underconfidence. Both theoretical analysis and empirical validation confirm that adherence to these principles significantly improves estimation accuracy, whereas fixed parameter choices commonly adopted in the literature are often suboptimal. The codes and the proofs for all the theorems in this paper are available at https://github.com/Shida-Jiang/Guidelines-for-Nonlinear-Kalman-Filters.

翻译：卡尔曼滤波器（KF）的非线性扩展，如扩展卡尔曼滤波器（EKF）和无迹卡尔曼滤波器（UKF），在复杂动力系统的状态估计中不可或缺，然而非线性KF提供稳健且精确估计的条件仍未被充分理解。本文提出一个理论框架，用于识别特定非线性KF成功与失败的原因，并建立其改进指南。该框架的核心概念是协方差补偿：即非线性KF预测的协方差与EKF协方差之间的偏差。基于这一定义及详细的理论分析，我们推导出非线性KF的三项设计指南：(i) 正交变换下的不变性；(ii) 超越EKF基线的充分协方差补偿；(iii) 偏向欠自信的补偿幅度选择。理论分析与实验验证均表明，遵循这些原则可显著提升估计精度，而文献中常用的固定参数选择往往非最优。本文所有定理的代码及证明见 https://github.com/Shida-Jiang/Guidelines-for-Nonlinear-Kalman-Filters。