The use of Large Language Models (LLMs) has drawn growing interest within the scientific community. LLMs can handle large volumes of textual data and support methods for evidence synthesis. Although recent studies highlight the potential of LLMs to accelerate screening and data extraction steps in systematic reviews, detailed reports of their practical application throughout the entire process remain scarce. This paper presents an experience report on the conduction of a systematic mapping study with the support of LLMs, describing the steps followed, the necessary adjustments, and the main challenges faced. Positive aspects are discussed, such as (i) the significant reduction of time in repetitive tasks and (ii) greater standardization in data extraction, as well as negative aspects, including (i) considerable effort to build reliable well-structured prompts, especially for less experienced users, since achieving effective prompts may require several iterations and testing, which can partially offset the expected time savings, (ii) the occurrence of hallucinations, and (iii) the need for constant manual verification. As a contribution, this work offers lessons learned and practical recommendations for researchers interested in adopting LLMs in systematic mappings and reviews, highlighting both efficiency gains and methodological risks and limitations to be considered.

翻译：大语言模型（LLMs）在科学界的应用日益受到关注。LLMs能够处理海量文本数据，并支持证据合成方法。尽管近期研究强调了LLMs在加速系统综述的筛选和数据提取步骤方面的潜力，但关于其在整个流程中实际应用的详细报告仍然匮乏。本文通过实践报告的形式，介绍了在LLMs支持下开展系统映射研究的经验，详细描述了实施步骤、必要调整以及面临的主要挑战。研究讨论了积极方面，例如：（i）重复性任务耗时显著减少；（ii）数据提取过程标准化程度提升；同时也指出了负面因素，包括：（i）构建可靠且结构良好的提示词需投入大量精力（尤其对经验不足的用户而言），因为实现有效提示可能需要多次迭代测试，这可能部分抵消预期的时间节省；（ii）幻觉现象的出现；（iii）需要持续的人工核查。作为本研究的贡献，我们为有意在系统映射与综述中采用LLMs的研究者提供了经验总结与实践建议，既阐明了效率提升的潜力，也指出了需要考虑的方法学风险与局限性。