This paper addresses learning end-to-end models for time series data that include a temporal alignment step via dynamic time warping (DTW). Existing approaches to differentiable DTW either differentiate through a fixed warping path or apply a differentiable relaxation to the min operator found in the recursive steps used to solve the DTW problem. We instead propose a DTW layer based around bi-level optimisation and deep declarative networks, which we name DecDTW. By formulating DTW as a continuous, inequality constrained optimisation problem, we can compute gradients for the solution of the optimal alignment (with respect to the underlying time series) using implicit differentiation. An interesting byproduct of this formulation is that DecDTW outputs the optimal warping path between two time series as opposed to a soft approximation, recoverable from Soft-DTW. We show that this property is particularly useful for applications where downstream loss functions are defined on the optimal alignment path itself. This naturally occurs, for instance, when learning to improve the accuracy of predicted alignments against ground truth alignments. We evaluate DecDTW on two such applications, namely the audio-to-score alignment task in music information retrieval and the visual place recognition task in robotics, demonstrating state-of-the-art results in both.

翻译：本文针对包含动态时间规整（DTW）时间对齐步骤的时间序列数据，研究其端到端模型的训练方法。现有可微DTW方法要么通过固定规整路径求导，要么对求解DTW递归步骤中的最小值运算符施加可微松弛。我们基于双层优化和深度声明式网络提出一种新的DTW层，命名为DecDTW。通过将DTW建模为连续不等式约束优化问题，我们可利用隐式微分计算最优对齐解（相对于底层时间序列）的梯度。该公式的有趣副产品在于：DecDTW输出的两个时间序列间的最优规整路径是确定性的，而非如Soft-DTW那样仅输出软近似结果。我们证明，当下游损失函数直接定义在最优对齐路径上时，这一特性尤为实用（例如当通过训练提升预测对齐结果相对于真实对齐结果的精度时，该场景天然存在）。我们在两个典型应用中评估DecDTW——音乐信息检索中的音频-乐谱对齐任务和机器人视觉地点识别任务，均取得了当时最优的成果。