This paper introduces an enormous dataset, HaGRID (HAnd Gesture Recognition Image Dataset), to build a hand gesture recognition (HGR) system concentrating on interaction with devices to manage them. That is why all 18 chosen gestures are endowed with the semiotic function and can be interpreted as a specific action. Although the gestures are static, they were picked up, especially for the ability to design several dynamic gestures. It allows the trained model to recognize not only static gestures such as "like" and "stop" but also "swipes" and "drag and drop" dynamic gestures. The HaGRID contains 554,800 images and bounding box annotations with gesture labels to solve hand detection and gesture classification tasks. The low variability in context and subjects of other datasets was the reason for creating the dataset without such limitations. Utilizing crowdsourcing platforms allowed us to collect samples recorded by 37,583 subjects in at least as many scenes with subject-to-camera distances from 0.5 to 4 meters in various natural light conditions. The influence of the diversity characteristics was assessed in ablation study experiments. Also, we demonstrate the HaGRID ability to be used for pretraining models in HGR tasks. The HaGRID and pretrained models are publicly available.
翻译:本文介绍了一个大规模数据集HaGRID(手势识别图像数据集),用于构建专注于设备交互控制的手势识别(HGR)系统。因此,所选的18种手势均具有符号功能,可被解释为特定操作指令。尽管手势均为静态手势,但基于设计多种动态手势的需求,本研究特别选取了这些手势。这使得训练后的模型不仅能识别"点赞""停止"等静态手势,还能识别"滑动""拖放"等动态手势。HaGRID包含554,800张图像及其边界框标注和手势标签,可解决手部检测和手势分类任务。针对现有数据集存在的场景与受试者多样性不足问题,本数据集突破了此类限制。通过众包平台,我们收集了37,583名受试者在至少同等数量的场景中,于0.5至4米距离范围内、多种自然光照条件下拍摄的样本。通过消融实验评估了多样性特征的影响,同时验证了HaGRID用于HGR任务模型预训练的可行性。该数据集及预训练模型均已公开提供。