Deaf individuals confront significant communication obstacles on a daily basis. Their inability to hear makes it difficult for them to communicate with those who do not understand sign language. Moreover, it presents difficulties in educational, occupational, and social contexts. By providing alternative communication channels, technology can play a crucial role in overcoming these obstacles. One such technology that can facilitate communication between deaf and hearing individuals is sign language recognition. We will create a robust system for sign language recognition in order to convert Indian Sign Language to text or speech. We will evaluate the proposed system and compare CNN and LSTM models. Since there are both static and gesture sign languages, a robust model is required to distinguish between them. In this study, we discovered that a CNN model captures letters and characters for recognition of static sign language better than an LSTM model, but it outperforms CNN by monitoring hands, faces, and pose in gesture sign language phrases and sentences. The creation of a text-to-sign language paradigm is essential since it will enhance the sign language-dependent deaf and hard-of-hearing population's communication skills. Even though the sign-to-text translation is just one side of communication, not all deaf or hard-of-hearing people are proficient in reading or writing text. Some may have difficulty comprehending written language due to educational or literacy issues. Therefore, a text-to-sign language paradigm would allow them to comprehend text-based information and participate in a variety of social, educational, and professional settings. Keywords: deaf and hard-of-hearing, DHH, Indian sign language, CNN, LSTM, static and gesture sign languages, text-to-sign language model, MediaPipe Holistic, sign language recognition, SLR, SLT
翻译:聋哑人在日常生活中面临重大的沟通障碍。由于无法听见声音,他们难以与不了解手语的人交流。此外,这还给他们在教育、职业和社交方面带来困难。通过提供替代性沟通渠道,技术可以在克服这些障碍方面发挥关键作用。手语识别就是这样一种能够促进聋哑人与健听人之间沟通的技术。我们将构建一个稳健的手语识别系统,以将印度手语转换为文本或语音。我们将评估所提出的系统,并比较CNN和LSTM模型。由于存在静态手语和手势手语,需要一个稳健的模型来区分它们。本研究发现,在静态手语识别中,CNN模型比LSTM模型能更好地捕捉字母和字符;但在基于手势的手语短语和句子识别中,LSTM通过监测手部、面部和姿态的表现优于CNN。文本到手语范式的创建至关重要,因为它将增强依赖手语的聋哑及听障人群的沟通能力。尽管手语到文本的翻译只是沟通的一个方面,但并非所有聋哑或听障人士都精通文本阅读或写作。由于教育或读写能力问题,有些人可能难以理解书面语言。因此,文本到手语范式将使他们能够理解基于文本的信息,并参与各种社交、教育和职业场合。关键词:聋哑及听障人士,DHH,印度手语,CNN,LSTM,静态与手势手语,文本到手语模型,MediaPipe Holistic,手语识别,SLR,SLT