Objective: This study aims to review the recent advances in community challenges for biomedical text mining in China. Methods: We collected information of evaluation tasks released in community challenges of biomedical text mining, including task description, dataset description, data source, task type and related links. A systematic summary and comparative analysis were conducted on various biomedical natural language processing tasks, such as named entity recognition, entity normalization, attribute extraction, relation extraction, event extraction, text classification, text similarity, knowledge graph construction, question answering, text generation, and large language model evaluation. Results: We identified 39 evaluation tasks from 6 community challenges that spanned from 2017 to 2023. Our analysis revealed the diverse range of evaluation task types and data sources in biomedical text mining. We explored the potential clinical applications of these community challenge tasks from a translational biomedical informatics perspective. We compared with their English counterparts, and discussed the contributions, limitations, lessons and guidelines of these community challenges, while highlighting future directions in the era of large language models. Conclusion: Community challenge evaluation competitions have played a crucial role in promoting technology innovation and fostering interdisciplinary collaboration in the field of biomedical text mining. These challenges provide valuable platforms for researchers to develop state-of-the-art solutions.
翻译:目的:本研究旨在综述中国生物医学文本挖掘领域社区挑战的最新进展。方法:我们收集了生物医学文本挖掘社区挑战中发布的评估任务信息,包括任务描述、数据集描述、数据来源、任务类型及相关链接。对命名实体识别、实体标准化、属性抽取、关系抽取、事件抽取、文本分类、文本相似度计算、知识图谱构建、问答系统、文本生成以及大语言模型评估等多种生物医学自然语言处理任务进行了系统性总结与对比分析。结果:我们从2017年至2023年间举办的6项社区挑战中识别出39个评估任务。分析揭示了生物医学文本挖掘评估任务类型和数据来源的多样性。我们从转化生物信息学视角探讨了这些社区挑战任务潜在的临床应用价值。通过与英文同类任务的对比,我们讨论了这些社区挑战的贡献、局限性、经验教训与实施规范,并重点展望了大语言模型时代的发展方向。结论:社区挑战评估竞赛在推动生物医学文本挖掘领域的技术创新和促进跨学科合作方面发挥了关键作用,为研究人员开发前沿解决方案提供了宝贵平台。