Reproducibility is a major concern across scientific fields. Human-Computer Interaction (HCI), in particular, is subject to diverse reproducibility challenges due to the wide range of research methodologies employed. In this article, we explore how the increasing adoption of Large Language Models (LLMs) across all user experience (UX) design and research activities impacts reproducibility in HCI. In particular, we review upcoming reproducibility challenges through the lenses of analogies from past to future (mis)practices like p-hacking and prompt-hacking, general bias, support in data analysis, documentation and education requirements, and possible pressure on the community. We discuss the risks and chances for each of these lenses with the expectation that a more comprehensive discussion will help shape best practices and contribute to valid and reproducible practices around using LLMs in HCI research.

翻译：可复现性是各科学领域关注的核心问题。由于人机交互（HCI）领域广泛采用多样化的研究方法，其面临着尤为突出的可复现性挑战。本文探讨了大型语言模型（LLMs）在用户体验（UX）设计与研究活动中的日益普及如何影响HCI领域的可复现性。我们通过过往至未来（不当）实践的类比视角（如p-hacking与prompt-hacking、普遍偏差、数据分析支持、文档记录与教育需求，以及可能对学术共同体造成的压力）审视即将面临的可复现性挑战。针对每个视角，我们讨论其风险与机遇，期望通过更全面的讨论助力形成最佳实践，并推动HCI研究中LLM应用的有效且可复现的研究范式。