Retrieving the biological impacts of protein-protein interactions (PPIs) is essential for target identification (Target ID) in drug development. Given the vast number of proteins involved, this process remains time-consuming and challenging. Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) frameworks have supported Target ID; however, no benchmark currently exists for identifying the biological impacts of PPIs. To bridge this gap, we introduce the RAG Benchmark for PPIs (RAGPPI), a factual question-answer benchmark of 4,420 question-answer pairs that focus on the potential biological impacts of PPIs. Through interviews with experts, we identified criteria for a benchmark dataset, such as a type of QA and source. We built a gold-standard dataset (500 QA pairs) through expert-driven data annotation. We developed an ensemble auto-evaluation LLM that incorporates expert labeling characteristics, average fact-abstract similarity (F1), and low-similarity fact counts (F2), enabling the construction of a silver-standard dataset (3,720 QA pairs). We are committed to maintaining RAGPPI as a resource to support the research community in advancing RAG systems for drug discovery QA solutions.

翻译：在药物开发中，检索蛋白质-蛋白质相互作用（PPIs）的生物学影响对于靶点识别（Target ID）至关重要。鉴于涉及的蛋白质数量庞大，这一过程仍然耗时且充满挑战。大型语言模型（LLMs）与检索增强生成（RAG）框架已为靶点识别提供支持，但目前尚无针对PPIs生物学影响识别的基准。为填补这一空白，我们提出了PPIs的RAG基准（RAGPPI），这是一个包含4,420个问答对的事实性问答基准，聚焦于PPIs潜在的生物学影响。通过专家访谈，我们确定了基准数据集的关键标准，例如问答类型和数据来源。我们基于专家驱动的数据标注构建了黄金标准数据集（500个问答对）。我们开发了一种集成自动评估LLM，该模型融合了专家标注特征、平均事实-摘要相似度（F1）以及低相似度事实计数（F2），从而构建了白银标准数据集（3,720个问答对）。我们致力于将RAGPPI作为资源持续维护，以支持研究社区推进面向药物发现问答解决方案的RAG系统。