Human activity recognition (HAR) with wearables is one of the serviceable technologies in ubiquitous and mobile computing applications. The sliding-window scheme is widely adopted while suffering from the multi-class windows problem. As a result, there is a growing focus on joint segmentation and recognition with deep-learning methods, aiming at simultaneously dealing with HAR and time-series segmentation issues. However, obtaining the full activity annotations of wearable data sequences is resource-intensive or time-consuming, while unsupervised methods yield poor performance. To address these challenges, we propose a novel method for joint activity segmentation and recognition with timestamp supervision, in which only a single annotated sample is needed in each activity segment. However, the limited information of sparse annotations exacerbates the gap between recognition and segmentation tasks, leading to sub-optimal model performance. Therefore, the prototypes are estimated by class-activation maps to form a sample-to-prototype contrast module for well-structured embeddings. Moreover, with the optimal transport theory, our approach generates the sample-level pseudo-labels that take advantage of unlabeled data between timestamp annotations for further performance improvement. Comprehensive experiments on four public HAR datasets demonstrate that our model trained with timestamp supervision is superior to the state-of-the-art weakly-supervised methods and achieves comparable performance to the fully-supervised approaches.

翻译：基于可穿戴设备的人体活动识别（HAR）是普适计算和移动计算应用中的关键技术之一。尽管滑动窗口方法被广泛采用，但其存在多类窗口问题。因此，联合分割与识别的深度学习方法日益受到关注，旨在同时解决HAR和时间序列分割问题。然而，为可穿戴数据序列获取完整的活动标注既耗时又耗资源，而无监督方法性能不佳。为应对这些挑战，我们提出了一种新颖的时间戳监督联合活动分割与识别方法，每个活动片段仅需一个标注样本。然而，稀疏标注信息有限，加剧了识别与分割任务之间的差距，导致模型性能次优。为此，我们通过类激活图估计原型，构建样本-原型对比模块以形成结构化良好的嵌入。此外，基于最优传输理论，我们的方法生成样本级伪标签，利用时间戳标注之间的未标注数据进一步改善性能。在四个公开HAR数据集上的全面实验表明，采用时间戳监督训练的模型优于现有最先进的弱监督方法，且性能与全监督方法相当。