This study demonstrates the application of instruction finetuning of pretrained Large Language Models (LLMs) to automate the generation of AI research leaderboards, extracting (Task, Dataset, Metric, Score) quadruples from articles. It aims to streamline the dissemination of advancements in AI research by transitioning from traditional, manual community curation, or otherwise taxonomy-constrained natural language inference (NLI) models, to an automated, generative LLM-based approach. Utilizing the FLAN-T5 model, this research enhances LLMs' adaptability and reliability in information extraction, offering a novel method for structured knowledge representation.

翻译：本研究展示了通过指令微调预训练大语言模型（LLMs）来自动化生成人工智能研究排行榜的应用，该方法从学术论文中提取（任务、数据集、指标、分数）四元组。其目标是通过从传统的人工社区整理、或受限于分类体系的自然语言推理（NLI）模型，转向一种自动化的、基于生成式LLM的方法，从而简化人工智能研究进展的传播。本研究利用FLAN-T5模型，提升了大语言模型在信息提取中的适应性和可靠性，为结构化知识表示提供了一种新方法。