We introduce an amortized variational inference algorithm and structured variational approximation for state-space models with nonlinear dynamics driven by Gaussian noise. Importantly, the proposed framework allows for efficient evaluation of the ELBO and low-variance stochastic gradient estimates without resorting to diagonal Gaussian approximations by exploiting (i) the low-rank structure of Monte-Carlo approximations to marginalize the latent state through the dynamics (ii) an inference network that approximates the update step with low-rank precision matrix updates (iii) encoding current and future observations into pseudo observations -- transforming the approximate smoothing problem into an (easier) approximate filtering problem. Overall, the necessary statistics and ELBO can be computed in $O(TL(Sr + S^2 + r^2))$ time where $T$ is the series length, $L$ is the state-space dimensionality, $S$ are the number of samples used to approximate the predict step statistics, and $r$ is the rank of the approximate precision matrix update in the update step (which can be made of much lower dimension than $L$).

翻译：我们针对由高斯噪声驱动的非线性动力学状态空间模型，提出了一种摊销变分推理算法及结构化变分近似。重要的是，该框架通过利用以下特性，能够在无需采用对角高斯近似的情况下，高效计算证据下界（ELBO）及低方差随机梯度估计：（i）蒙特卡罗近似的低秩结构，通过动力学对潜在状态进行边缘化；（ii）一种推理网络，以低秩精度矩阵更新近似更新步骤；（iii）将当前及未来观测编码为伪观测——将平滑问题转化为更易处理的滤波问题。总体而言，所需统计量与ELBO可在$O(TL(Sr + S^2 + r^2))$时间内计算完成，其中$T$为时序长度，$L$为状态空间维度，$S$为用于近似预测步骤统计量的样本数量，$r$为更新步骤中近似精度矩阵更新的秩（其维度可远低于$L$）。