Large language models (LLMs) produce context inconsistency hallucinations, which are LLM generated outputs that are misaligned with the user prompt. This research project investigates whether prompt engineering (PE) methods can mitigate context inconsistency hallucinations in zero-shot LLM summarisation of scientific texts, where zero-shot indicates that the LLM relies purely on its pre-training data. Across eight yeast biotechnology research paper abstracts, six instruction-tuned LLMs were prompted with seven methods: a base- line prompt, two levels of increasing instruction complexity (PE-1 and PE-2), two levels of context repetition (CR-K1 and CR-K2), and two levels of random addition (RA-K1 and RA-K2). Context repetition involved the identification and repetition of K key sentences from the abstract, whereas random addition involved the repetition of K randomly selected sentences from the abstract, where K is 1 or 2. A total of 336 LLM-generated summaries were evaluated using six metrics: ROUGE-1, ROUGE-2, ROUGE-L, BERTScore, METEOR, and cosine similarity, which were used to compute the lexical and semantic alignment be- tween the summaries and the abstracts. Four hypotheses on the effects of prompt methods on summary alignment with the reference text were tested. Statistical analysis on 3744 collected datapoints was performed using bias-corrected and accelerated (BCa) bootstrap confidence intervals and Wilcoxon signed-rank tests with Bonferroni-Holm correction. The results demonstrated that CR and RA significantly improve the lexical alignment of LLM-generated summaries with the abstracts. These findings indicate that prompt engineering has the potential to impact hallucinations in zero-shot scientific summarisation tasks.

翻译：大语言模型（LLMs）会产生上下文不一致的幻觉，即LLM生成的输出与用户提示不一致。本研究项目探讨了提示工程（PE）方法是否能够缓解零样本LLM科学文本摘要中的上下文不一致幻觉，其中零样本指LLM完全依赖其预训练数据。基于八篇酵母生物技术研究论文摘要，对六个指令调优的LLM使用了七种提示方法：基线提示、两种递增指令复杂度方法（PE-1和PE-2）、两种上下文重复方法（CR-K1和CR-K2）以及两种随机添加方法（RA-K1和RA-K2）。上下文重复涉及识别并重复摘要中的K个关键句子，而随机添加则重复摘要中随机选择的K个句子，其中K为1或2。使用六种指标对总共336个LLM生成的摘要进行了评估：ROUGE-1、ROUGE-2、ROUGE-L、BERTScore、METEOR和余弦相似度，这些指标用于计算摘要与摘要之间的词汇和语义对齐度。测试了关于提示方法对摘要与参考文本对齐度影响的四个假设。对收集的3744个数据点进行了统计分析，使用了偏差校正加速（BCa）自助法置信区间和经Bonferroni-Holm校正的Wilcoxon符号秩检验。结果表明，CR和RA显著提高了LLM生成摘要与摘要的词汇对齐度。这些发现表明，提示工程有潜力影响零样本科学摘要任务中的幻觉现象。