This work presents a novel approach called oracle-checker scheme for evaluating the answer given by a generative large language model (LLM). Two types of checkers are presented. The first type of checker follows the idea of property testing. The second type of checker follows the idea of program checking. Their applications are demonstrated in two separate contexts, entity extraction and paraphrase decision, respectively.

翻译：本文提出了一种名为Oracle-Checker的新方案，用于评估生成式大语言模型（LLM）给出的答案。该方案包含两种类型的检查器：第一种遵循属性测试的思想，第二种遵循程序检查的思想。分别以实体抽取和释义判定两个独立场景为例，展示了其应用。