The lack of fluency in sign language remains a barrier to seamless communication for hearing and speech-impaired communities. In this work, we propose a low-cost, real-time ASL-to-speech translation glove and an exhaustive training dataset of sign language patterns. We then benchmarked this dataset with supervised learning models, such as LSTMs, GRUs and Transformers, where our best model achieved 92% accuracy. The SignSpeak dataset has 7200 samples encompassing 36 classes (A-Z, 1-10) and aims to capture realistic signing patterns by using five low-cost flex sensors to measure finger positions at each time step at 36 Hz. Our open-source dataset, models and glove designs, provide an accurate and efficient ASL translator while maintaining cost-effectiveness, establishing a framework for future work to build on.
翻译:手语不流利仍是听障与言语障碍群体实现无缝沟通的障碍。本研究提出一种低成本、实时性的美国手语(ASL)到语音翻译手套,并构建了涵盖手语模式的详尽训练数据集。随后,我们使用LSTM、GRU和Transformer等监督学习模型对该数据集进行基准测试,其中最优模型准确率达到92%。SignSpeak数据集包含7200个样本,涵盖36个类别(A-Z、1-10),通过五个低成本弯曲传感器以36Hz频率测量每个时间步的手指位置,旨在捕捉真实的手语模式。我们开源的数据集、模型及手套设计在保持成本效益的同时,提供了精准高效的ASL翻译方案,为后续研究建立了可扩展的框架。