We investigate intelligent personal assistants (IPAs) accessibility for deaf and hard of hearing (DHH) people who can use their voice in everyday communication. The inability of IPAs to understand diverse accents including deaf speech renders them largely inaccessible to non-signing and speaking DHH individuals. Using an Echo Show, we compare the usability of natural language input via spoken English; with Alexa's automatic speech recognition and a Wizard-of-Oz setting with a trained facilitator re-speaking commands against that of a large language model (LLM)-assisted touch interface in a mixed-methods study. The touch method was navigated through an LLM-powered "task prompter," which integrated the user's history and smart environment to suggest contextually-appropriate commands. Quantitative results showed no significant differences across both spoken English conditions vs LLM-assisted touch. Qualitative results showed variability in opinions on the usability of each method. Ultimately, it will be necessary to have robust deaf-accented speech recognized natively by IPAs.

翻译：本研究探讨了能够在日常交流中使用语音的聋人与听障人士对智能个人助理的可访问性。由于智能个人助理无法理解包括聋人语音在内的多样化口音，导致其对于非手语使用者但具备口语能力的聋人与听障个体基本不可用。通过采用混合研究方法，我们使用Echo Show设备比较了以下三种交互方式的可用性：基于英语自然语言的语音输入（通过Alexa自动语音识别系统实现）、采用训练有素的协助者复述指令的“绿野仙踪”模拟设置，以及基于大语言模型的触控界面。触控交互方式通过一个由大语言模型驱动的“任务提示器”进行导航，该提示器整合用户历史记录与智能环境信息，以推荐符合情境的操作指令。定量研究结果显示，两种英语语音交互条件与基于大语言模型的触控界面之间不存在显著差异。定性研究结果则显示参与者对不同交互方式的可用性评价存在差异。最终，实现智能个人助理原生支持对聋人口音的鲁棒性语音识别将是必要的发展方向。