Real-time detection of out-of-context LLM outputs is crucial for enterprises looking to safely adopt RAG applications. In this work, we train lightweight models to discriminate LLM-generated text that is semantically out-of-context from retrieved text documents. We preprocess a combination of summarisation and semantic textual similarity datasets to construct training data using minimal resources. We find that DeBERTa is not only the best-performing model under this pipeline, but it is also fast and does not require additional text preprocessing or feature engineering. While emerging work demonstrates that generative LLMs can also be fine-tuned and used in complex data pipelines to achieve state-of-the-art performance, we note that speed and resource limits are important considerations for on-premise deployment.

翻译：实时检测大型语言模型（LLM）生成的上下文外输出对于企业安全采用检索增强生成（RAG）应用至关重要。本研究通过训练轻量级模型，以判别LLM生成的文本是否与检索到的文本文档存在语义上的上下文偏离。我们通过整合摘要生成与语义文本相似性数据集，以最小资源消耗构建训练数据。研究发现，DeBERTa不仅在该流程中表现最优，且具备高速处理能力，无需额外的文本预处理或特征工程。尽管新兴研究表明生成式LLM也可通过微调并应用于复杂数据流程以实现最先进性能，但我们强调处理速度与资源限制对于本地化部署具有重要考量意义。