Sign language discourse is an essential mode of daily communication for the deaf and hard-of-hearing people. However, research on Bangla Sign Language (BdSL) faces notable limitations, primarily due to the lack of datasets. Recognizing wordlevel signs in BdSL (WL-BdSL) presents a multitude of challenges, including the need for well-annotated datasets, capturing the dynamic nature of sign gestures from facial or hand landmarks, developing suitable machine learning or deep learning-based models with substantial video samples, and so on. In this paper, we address these challenges by creating a comprehensive BdSL word-level dataset named BdSLW60 in an unconstrained and natural setting, allowing positional and temporal variations and allowing sign users to change hand dominance freely. The dataset encompasses 60 Bangla sign words, with a significant scale of 9307 video trials provided by 18 signers under the supervision of a sign language professional. The dataset was rigorously annotated and cross-checked by 60 annotators. We also introduced a unique approach of a relative quantization-based key frame encoding technique for landmark based sign gesture recognition. We report the benchmarking of our BdSLW60 dataset using the Support Vector Machine (SVM) with testing accuracy up to 67.6% and an attention-based bi-LSTM with testing accuracy up to 75.1%. The dataset is available at https://www.kaggle.com/datasets/hasaniut/bdslw60 and the code base is accessible from https://github.com/hasanssl/BdSLW60_Code.
翻译:手语交流是聋哑及听障人群日常沟通的重要方式。然而,针对孟加拉手语的研究面临显著局限性,主要在于数据集的匮乏。词级孟加拉手语的识别面临诸多挑战,包括需要高质量标注的数据集、从面部或手部关键点捕捉手势的动态特性、开发基于大量视频样本的机器学习或深度学习模型等。本文通过构建一个名为BdSLW60的综合性孟加拉手语词级数据集,在无约束自然环境下解决了这些挑战,允许位置和时间上的变化,并让手语用户自由切换主导手。该数据集包含60个孟加拉手语词汇,规模达9307个视频样本,由18位手语者在专业手语人士监督下完成。数据集经过60位标注员的严格标注和交叉验证。我们创新性地提出了一种基于相对量化的关键帧编码技术,用于基于关键点的手势识别。我们使用支持向量机(测试准确率最高达67.6%)和基于注意力的双向长短期记忆网络(测试准确率最高达75.1%)对BdSLW60数据集进行了基准测试。数据集可从https://www.kaggle.com/datasets/hasaniut/bdslw60 获取,代码库可从https://github.com/hasanssl/BdSLW60_Code 访问。