For training a video-based action recognition model that accepts multi-view video, annotating frame-level labels is tedious and difficult. However, it is relatively easy to annotate sequence-level labels. This kind of coarse annotations are called as weak labels. However, training a multi-view video-based action recognition model with weak labels for frame-level perception is challenging. In this paper, we propose a novel learning framework, where the weak labels are first used to train a multi-view video-based base model, which is subsequently used for downstream frame-level perception tasks. The base model is trained to obtain individual latent embeddings for each view in the multi-view input. For training the model using the weak labels, we propose a novel latent loss function. We also propose a model that uses the view-specific latent embeddings for downstream frame-level action recognition and detection tasks. The proposed framework is evaluated using the MM Office dataset by comparing several baseline algorithms. The results show that the proposed base model is effectively trained using weak labels and the latent embeddings help the downstream models improve accuracy.

翻译：对于训练接受多视角视频作为输入的视频动作识别模型而言，标注帧级标签既繁琐又困难。然而，标注序列级标签相对容易，这种粗粒度的标注被称为弱标签。然而，利用弱标签训练用于帧级感知的多视角视频动作识别模型颇具挑战性。本文提出一种新颖的学习框架：首先使用弱标签训练一个基于多视角视频的基础模型，随后将该模型用于下游的帧级感知任务。基础模型通过训练获取多视角输入中每个视角的独立潜在嵌入表示。针对使用弱标签训练模型的问题，我们提出了一种新颖的潜在损失函数。同时，我们还提出了一个利用视角特定潜在嵌入来执行下游帧级动作识别与检测任务的模型。通过在MM Office数据集上对比若干基线算法，对所提框架进行了评估。结果表明，所提出的基础模型能够有效利用弱标签进行训练，且潜在嵌入有助于提升下游模型的准确率。