We present a novel local-global feature fusion framework for body-weight exercise recognition with floor-based dynamic pressure maps. One step further from the existing studies using deep neural networks mainly focusing on global feature extraction, the proposed framework aims to combine local and global features using image processing techniques and the YOLO object detection to localize pressure profiles from different body parts and consider physical constraints. The proposed local feature extraction method generates two sets of high-level local features consisting of cropped pressure mapping and numerical features such as angular orientation, location on the mat, and pressure area. In addition, we adopt a knowledge distillation for regularization to preserve the knowledge of the global feature extraction and improve the performance of the exercise recognition. Our experimental results demonstrate a notable 11 percent improvement in F1 score for exercise recognition while preserving label-specific features.

翻译：我们提出了一种新颖的局部-全局特征融合框架，用于基于地面动态压力图的体重锻炼识别。与现有主要关注全局特征提取的深度神经网络研究相比，本框架旨在结合图像处理技术与YOLO目标检测方法，通过定位不同身体部位的压力分布并考虑物理约束来实现局部与全局特征的融合。所提出的局部特征提取方法生成两组高层局部特征：裁剪压力映射图与数值特征（包括角度朝向、垫上位置及压力面积）。此外，我们采用知识蒸馏作为正则化手段，以保留全局特征提取的知识并提升锻炼识别性能。实验结果显示，在保持标签特定特征的前提下，锻炼识别的F1分数提升了11%。