Reproducibility is a major concern across scientific fields. Human-Computer Interaction (HCI), in particular, is subject to diverse reproducibility challenges due to the wide range of research methodologies employed. In this article, we explore how the increasing adoption of Large Language Models (LLMs) across all user experience (UX) design and research activities impacts reproducibility in HCI. In particular, we review upcoming reproducibility challenges through the lenses of analogies from past to future (mis)practices like p-hacking and prompt-hacking, general bias, support in data analysis, documentation and education requirements, and possible pressure on the community. We discuss the risks and chances for each of these lenses with the expectation that a more comprehensive discussion will help shape best practices and contribute to valid and reproducible practices around using LLMs in HCI research.

翻译：可复现性是科学领域普遍关注的重要问题。人机交互（HCI）领域尤其面临多样化的可复现性挑战，这源于其所采用的研究方法范围广泛。本文探讨了在用户体验（UX）设计与研究活动中日益广泛采用的大语言模型（LLMs）如何影响HCI领域的可复现性。具体而言，我们通过类比过去与未来可能出现的（不当）实践（如p-hacking与提示词操纵）、普遍偏见、数据分析支持、文档与教育要求，以及对学术共同体可能产生的压力等视角，审视了即将出现的可复现性挑战。我们针对每个视角讨论了其风险与机遇，期望通过更全面的讨论有助于形成最佳实践，并为在HCI研究中有效且可复现地使用LLMs做出贡献。