Automatic Speech Recognition (ASR) is traditionally evaluated using Word Error Rate (WER), a metric that is insensitive to meaning. Embedding-based semantic metrics are better correlated with human perception, but decoder-based Large Language Models (LLMs) remain underexplored for this task. This paper evaluates their relevance through three approaches: (1) selecting the best hypothesis between two candidates, (2) computing semantic distance using generative embeddings, and (3) qualitative classification of errors. On the HATS dataset, the best LLMs achieve 92--94\% agreement with human annotators for hypothesis selection, compared to 63\% for WER, also outperforming semantic metrics. Embeddings from decoder-based LLMs show performance comparable to encoder models. Finally, LLMs offer a promising direction for interpretable and semantic ASR evaluation.

翻译：自动语音识别（ASR）传统上采用词错误率（WER）进行评估，但该指标对语义不敏感。基于嵌入的语义指标与人类感知的相关性更佳，然而解码器型大语言模型（LLM）在此任务上的应用尚未被充分探索。本文通过三种方法评估其有效性：（1）从两个候选结果中选择最优假设，（2）利用生成式嵌入计算语义距离，（3）对错误进行定性分类。在HATS数据集上，最优LLM在假设选择任务中与人类标注者的一致性达到92-94%，而基于WER的方法仅为63%，同时该结果也优于语义指标。解码器型LLM的嵌入表现可与编码器模型相媲美。最后，LLM为具备可解释性的语义级ASR评测提供了前景方向。