The development of robust, generalized models in human activity recognition (HAR) has been hindered by the scarcity of large-scale, labeled data sets. Recent work has shown that virtual IMU data extracted from videos using computer vision techniques can lead to substantial performance improvements when training HAR models combined with small portions of real IMU data. Inspired by recent advances in motion synthesis from textual descriptions and connecting Large Language Models (LLMs) to various AI models, we introduce an automated pipeline that first uses ChatGPT to generate diverse textual descriptions of activities. These textual descriptions are then used to generate 3D human motion sequences via a motion synthesis model, T2M-GPT, and later converted to streams of virtual IMU data. We benchmarked our approach on three HAR datasets (RealWorld, PAMAP2, and USC-HAD) and demonstrate that the use of virtual IMU training data generated using our new approach leads to significantly improved HAR model performance compared to only using real IMU data. Our approach contributes to the growing field of cross-modality transfer methods and illustrate how HAR models can be improved through the generation of virtual training data that do not require any manual effort.

翻译：人体活动识别（HAR）中鲁棒泛化模型的开发长期受限于大规模标注数据集的匮乏。近期研究表明，通过计算机视觉技术从视频中提取虚拟惯性测量单元（IMU）数据，在结合少量真实IMU数据训练HAR模型时，可显著提升模型性能。受文本描述驱动运动合成技术以及将大型语言模型（LLMs）与各类AI模型相连接的最新进展启发，我们提出了一种自动化流水线：首先利用ChatGPT生成多样化的活动文本描述，随后通过运动合成模型T2M-GPT将文本描述转化为三维人体运动序列，最终转换为虚拟IMU数据流。我们在三个HAR数据集（RealWorld、PAMAP2和USC-HAD）上进行了基准测试，结果表明，相较于仅使用真实IMU数据，采用本方法生成的虚拟IMU训练数据可显著提升HAR模型性能。该研究为跨模态迁移方法领域贡献了新的思路，揭示了无需人工标注即可通过生成虚拟训练数据改进HAR模型的可行路径。