Large language models (LLMs) have been positioned to revolutionize HCI, by reshaping not only the interfaces, design patterns, and sociotechnical systems that we study, but also the research practices we use. To-date, however, there has been little understanding of LLMs' uptake in HCI. We address this gap via a systematic literature review of 153 CHI papers from 2020-24 that engage with LLMs. We taxonomize: (1) domains where LLMs are applied; (2) roles of LLMs in HCI projects; (3) contribution types; and (4) acknowledged limitations and risks. We find LLM work in 10 diverse domains, primarily via empirical and artifact contributions. Authors use LLMs in five distinct roles, including as research tools or simulated users. Still, authors often raise validity and reproducibility concerns, and overwhelmingly study closed models. We outline opportunities to improve HCI research with and on LLMs, and provide guiding questions for researchers to consider the validity and appropriateness of LLM-related work.

翻译：大型语言模型（LLM）被定位为将彻底改变人机交互领域，其不仅重塑了我们所研究的交互界面、设计模式与社会技术系统，也变革了我们采用的研究实践。然而迄今为止，学界对LLM在人机交互领域的应用程度仍缺乏系统认知。本研究通过系统性文献综述填补了这一空白，分析了2020-2024年间涉及LLM的153篇CHI会议论文。我们构建了以下分类体系：(1) LLM的应用领域；(2) LLM在人机交互项目中的角色定位；(3) 研究成果类型；(4) 已认知的局限性与风险。研究发现LLM研究覆盖10个不同领域，主要通过实证研究与系统构建两类成果呈现。作者将LLM应用于五种不同角色，包括作为研究工具或模拟用户。尽管如此，研究者常提出效度与可复现性方面的担忧，且绝大多数研究聚焦于闭源模型。本文最后提出改进LLM相关人机交互研究的机遇，并为研究者评估LLM相关工作的效度与适用性提供了指导性问题框架。