This study presents a novel zero-shot user-defined keyword spotting model that utilizes the audio-phoneme relationship of the keyword to improve performance. Unlike the previous approach that estimates at utterance level, we use both utterance and phoneme level information. Our proposed method comprises a two-stream speech encoder architecture, self-attention-based pattern extractor, and phoneme-level detection loss for high performance in various pronunciation environments. Based on experimental results, our proposed model outperforms the baseline model and achieves competitive performance compared with full-shot keyword spotting models. Our proposed model significantly improves the EER and AUC across all datasets, including familiar words, proper nouns, and indistinguishable pronunciations, with an average relative improvement of 67% and 80%, respectively. The implementation code of our proposed model is available at https://github.com/ncsoft/PhonMatchNet.

翻译：本研究提出了一种新颖的零样本人定义关键词唤醒模型，该模型利用关键词的音频-音素对应关系来提升性能。不同于此前在语句层面进行估计的方法，我们同时使用语句和音素层面的信息。所提方法包含双流语音编码器架构、基于自注意力的模式提取器以及音素级检测损失函数，从而在各种发音环境下实现高性能。基于实验结果，我们的模型优于基线模型，并与全样本关键词唤醒模型相比具有竞争力的性能。该模型在所有数据集（包括常见词汇、专有名词及难以区分的发音）上显著提升了等错误率和AUC指标，平均相对改进幅度分别达到67%和80%。所提模型的实现代码已开源在https://github.com/ncsoft/PhonMatchNet。