This paper presents our contribution to the MEDIQA-2023 Dialogue2Note shared task, encompassing both subtask A and subtask B. We approach the task as a dialogue summarization problem and implement two distinct pipelines: (a) a fine-tuning of a pre-trained dialogue summarization model and GPT-3, and (b) few-shot in-context learning (ICL) using a large language model, GPT-4. Both methods achieve excellent results in terms of ROUGE-1 F1, BERTScore F1 (deberta-xlarge-mnli), and BLEURT, with scores of 0.4011, 0.7058, and 0.5421, respectively. Additionally, we predict the associated section headers using RoBERTa and SciBERT based classification models. Our team ranked fourth among all teams, while each team is allowed to submit three runs as part of their submission. We also utilize expert annotations to demonstrate that the notes generated through the ICL GPT-4 are better than all other baselines. The code for our submission is available.
翻译:本文介绍了我们对MEDIQA-2023 Dialogue2Note共享任务(包含子任务A和子任务B)的贡献。我们将该任务视为对话摘要问题,并实现了两种不同的流程:(a) 对预训练的对话摘要模型和GPT-3进行微调;(b) 使用大型语言模型GPT-4进行少样本上下文学习(ICL)。两种方法在ROUGE-1 F1、BERTScore F1(deberta-xlarge-mnli)和BLEURT指标上均取得了优异结果,得分分别为0.4011、0.7058和0.5421。此外,我们使用基于RoBERTa和SciBERT的分类模型预测关联的章节标题。我们的团队在所有参赛团队中排名第四(每个团队允许提交三个运行版本)。我们还利用专家标注证明,通过ICL GPT-4生成的笔记优于所有其他基线方法。本工作的代码已公开提供。