In this paper, we investigate building a sequence to sequence architecture for motion to language translation and synchronization. The aim is to translate motion capture inputs into English natural-language descriptions, such that the descriptions are generated synchronously with the actions performed, enabling semantic segmentation as a byproduct, but without requiring synchronized training data. We propose a new recurrent formulation of local attention that is suited for synchronous/live text generation, as well as an improved motion encoder architecture better suited to smaller data and for synchronous generation. We evaluate both contributions in individual experiments, using the standard BLEU4 metric, as well as a simple semantic equivalence measure, on the KIT motion language dataset. In a follow-up experiment, we assess the quality of the synchronization of generated text in our proposed approaches through multiple evaluation metrics. We find that both contributions to the attention mechanism and the encoder architecture additively improve the quality of generated text (BLEU and semantic equivalence), but also of synchronization. Our code will be made available at \url{https://github.com/rd20karim/M2T-Segmentation/tree/main}

翻译：本文研究了构建用于运动到语言翻译与同步的序列到序列架构。目标是将来运动捕捉输入翻译为英文自然语言描述，使得描述与所执行的动作同步生成，并作为副产品实现语义分割，但无需同步训练数据。我们提出了一种新的循环局部注意机制，适用于同步/实时文本生成，并改进了运动编码器架构，使其更适应小规模数据及同步生成。通过标准BLEU4指标及一种简单的语义等价度量，在KIT运动语言数据集上分别对这两项贡献进行了实验评估。在后续实验中，我们通过多种评估指标衡量所提方法生成文本的同步质量。研究发现，注意机制与编码器架构的贡献均能叠加性地提升生成文本的质量（BLEU与语义等价性），同时改善同步性能。我们的代码将发布于\url{https://github.com/rd20karim/M2T-Segmentation/tree/main}。