This paper describes our submission to the MEDIQA-CORR 2024 shared task for automatically detecting and correcting medical errors in clinical notes. We report results for three methods of few-shot In-Context Learning (ICL) augmented with Chain-of-Thought (CoT) and reason prompts using a large language model (LLM). In the first method, we manually analyse a subset of train and validation dataset to infer three CoT prompts by examining error types in the clinical notes. In the second method, we utilise the training dataset to prompt the LLM to deduce reasons about their correctness or incorrectness. The constructed CoTs and reasons are then augmented with ICL examples to solve the tasks of error detection, span identification, and error correction. Finally, we combine the two methods using a rule-based ensemble method. Across the three sub-tasks, our ensemble method achieves a ranking of 3rd for both sub-task 1 and 2, while securing 7th place in sub-task 3 among all submissions.
翻译:本文介绍了我们为MEDIQA-CORR 2024共享任务提交的系统,该任务旨在自动检测并纠正临床记录中的医疗错误。我们报告了三种基于大语言模型(LLM)的少样本上下文学习方法,这些方法均通过思维链提示和推理提示进行增强。在第一种方法中,我们通过人工分析训练集与验证集的子集,根据临床记录中的错误类型归纳出三种思维链提示模板。第二种方法则利用训练数据集提示大语言模型,推导出关于文本正确性与错误的判断依据。所构建的思维链与推理依据随后与少样本示例结合,共同完成错误检测、错误范围定位及错误纠正三项子任务。最后,我们通过基于规则的集成方法将两种方法进行融合。在所有提交系统中,我们的集成方法在子任务1和子任务2中均位列第三,在子任务3中排名第七。