The wide adoption of Large language models (LLMs) makes their dependability a pressing concern. Detection of errors is the first step to mitigating their impact on a system and thus, efficient error detection for LLMs is an important issue. In many settings, the LLM is considered as a black box with no access to the internal nodes; this prevents the use of many error detection schemes that need access to the model's internal nodes. An interesting observation is that the output of LLMs in error-free operation should be valid and normal text. Therefore, when the text is not valid or differs significantly from normal text, it is likely that there is an error. Based on this observation we propose to perform Concurrent Linguistic Error Detection (CLED); this scheme extracts some linguistic features of the text generated by the LLM and feeds them to a concurrent classifier that detects errors. Since the proposed error detection mechanism only relies on the outputs of the model, then it can be used on LLMs in which there is no access to the internal nodes. The proposed CLED scheme has been evaluated on the T5 model when used for news summarization and on the OPUS-MT model when used for translation. In both cases, the same set of linguistic features has been used for error detection to illustrate the applicability of the proposed scheme beyond a specific case. The results show that CLED can detect most of the errors at a low overhead penalty. The use of the concurrent classifier also enables a trade-off between error detection effectiveness and its associated overhead, so providing flexibility to a designer.

翻译：大规模语言模型（LLM）的广泛应用使其可靠性成为亟待解决的问题。错误检测是减轻其系统影响的首要步骤，因此针对LLM的高效错误检测具有重要研究价值。在多数应用场景中，LLM被视为无法访问内部节点的黑箱模型，这限制了需要访问模型内部节点的错误检测方案的使用。一个有趣的发现是：LLM在无错误运行时的输出应为有效且规范的文本。因此，当生成的文本不符合规范或与正常文本存在显著差异时，很可能存在错误。基于这一观察，我们提出并发语言错误检测（CLED）方案。该方法通过提取LLM生成文本的语言特征，将其输入并发分类器进行错误识别。由于所提出的错误检测机制仅依赖模型输出，因此适用于无法访问内部节点的LLM系统。我们分别在新闻摘要场景的T5模型与翻译场景的OPUS-MT模型上验证了CLED方案的有效性。两种场景均采用相同的语言特征集进行错误检测，以证明该方法在特定应用场景之外的通用性。实验结果表明，CLED能以较低开销检测出大部分错误。同时，并发分类器机制支持在错误检测效率与系统开销之间进行权衡，为设计者提供了灵活的配置空间。