This paper introduces an approach that combines the language reasoning capabilities of large language models (LLMs) with the benefits of local training to tackle complex, domain-specific tasks. Specifically, the authors demonstrate their approach by extracting structured condition codes from pathology reports. The proposed approach utilizes local LLMs, which can be fine-tuned to respond to specific generative instructions and provide structured outputs. The authors collected a dataset of over 150k uncurated surgical pathology reports, containing gross descriptions, final diagnoses, and condition codes. They trained different model architectures, including LLaMA, BERT and LongFormer and evaluated their performance. The results show that the LLaMA-based models significantly outperform BERT-style models across all evaluated metrics, even with extremely reduced precision. The LLaMA models performed especially well with large datasets, demonstrating their ability to handle complex, multi-label tasks. Overall, this work presents an effective approach for utilizing LLMs to perform domain-specific tasks using accessible hardware, with potential applications in the medical domain, where complex data extraction and classification are required.

翻译：本文提出了一种结合大语言模型（LLMs）语言推理能力与本地训练优势的方法，以解决特定领域的复杂任务。具体而言，作者通过从病理报告中提取结构化条件代码来演示其方法。所提方法利用本地大语言模型，该模型可通过微调响应特定生成指令并提供结构化输出。作者收集了超过15万份未经整理的外科病理报告数据集，包含大体描述、最终诊断和条件代码。他们训练了包括LLaMA、BERT和LongFormer在内的不同模型架构，并评估了其性能。结果表明，即便在极低精度下，基于LLaMA的模型在所有评估指标上均显著优于BERT类模型。LLaMA模型在大数据集上表现尤为出色，展现了其处理复杂多标签任务的能力。总体而言，本研究提出了一种利用大语言模型在可访问硬件上执行领域特定任务的有效方法，在需要复杂数据提取与分类的医学领域具有潜在应用价值。