Sensor-based human activity segmentation and recognition are two important and challenging problems in many real-world applications and they have drawn increasing attention from the deep learning community in recent years. Most of the existing deep learning works were designed based on pre-segmented sensor streams and they have treated activity segmentation and recognition as two separate tasks. In practice, performing data stream segmentation is very challenging. We believe that both activity segmentation and recognition may convey unique information which can complement each other to improve the performance of the two tasks. In this paper, we firstly proposes a new multitask deep neural network to solve the two tasks simultaneously. The proposed neural network adopts selective convolution and features multiscale windows to segment activities of long or short time durations. First, multiple windows of different scales are generated to center on each unit of the feature sequence. Then, the model is trained to predict, for each window, the activity class and the offset to the true activity boundaries. Finally, overlapping windows are filtered out by non-maximum suppression, and adjacent windows of the same activity are concatenated to complete the segmentation task. Extensive experiments were conducted on eight popular benchmarking datasets, and the results show that our proposed method outperforms the state-of-the-art methods both for activity recognition and segmentation.

翻译：传感器人体活动分割与识别是许多实际应用中两个重要且具有挑战性的问题，近年来已引起深度学习领域的广泛关注。现有的大多数深度学习工作基于预分割的传感器数据流设计，将活动分割与识别视为两个独立的任务。实践中，执行数据流分割极具挑战性。我们认为活动分割与识别可能传递独特信息，两者可互补以提升各自任务性能。本文首次提出一种新型多任务深度神经网络，用于同步求解这两类任务。该网络采用选择性卷积并具备多尺度窗口特性，能够分割长短期活动。首先，生成多个不同尺度的窗口，使其以特征序列的每个单元为中心。随后，训练模型对每个窗口预测活动类别以及到真实活动边界的偏移量。最终，通过非极大值抑制滤除重叠窗口，并将相同活动的相邻窗口拼接以完成分割任务。在八个公开基准数据集上的大量实验表明，本文提出的方法在活动识别与分割任务上均优于现有最优方法。