The ubiquity of large-scale Pre-Trained Models (PTMs) is on the rise, sparking interest in model hubs, and dedicated platforms for hosting PTMs. Despite this trend, a comprehensive exploration of the challenges that users encounter and how the community leverages PTMs remains lacking. To address this gap, we conducted an extensive mixed-methods empirical study by focusing on discussion forums and the model hub of HuggingFace, the largest public model hub. Based on our qualitative analysis, we present a taxonomy of the challenges and benefits associated with PTM reuse within this community. We then conduct a quantitative study to track model-type trends and model documentation evolution over time. Our findings highlight prevalent challenges such as limited guidance for beginner users, struggles with model output comprehensibility in training or inference, and a lack of model understanding. We also identified interesting trends among models where some models maintain high upload rates despite a decline in topics related to them. Additionally, we found that despite the introduction of model documentation tools, its quantity has not increased over time, leading to difficulties in model comprehension and selection among users. Our study sheds light on new challenges in reusing PTMs that were not reported before and we provide recommendations for various stakeholders involved in PTM reuse.

翻译：大规模预训练模型的普及程度日益提升，激发了人们对模型中心以及专门托管预训练模型平台的研究兴趣。然而，目前仍缺乏对用户面临的挑战以及社区如何利用预训练模型的全面探索。为填补这一空白，我们采用混合研究方法，针对HuggingFace（最大的公共模型中心）的讨论论坛和模型中心开展了广泛的实证研究。基于定性分析，我们提出了该社区内预训练模型复用所涉挑战与益处的分类体系。随后通过定量研究追踪模型类型趋势及模型文档随时间演变的规律。研究发现包括：入门用户指导有限、模型训练或推理输出结果的可理解性问题突出、以及模型理解不足等常见挑战。我们还观察到模型间的有趣趋势——部分模型虽相关话题热度下降，但上传速率仍维持高位。此外，尽管模型文档工具已投入使用，其数量并未随时间增长，导致用户在模型理解与选择方面存在困难。本研究揭示了先前未被报道的预训练模型复用新挑战，并为涉及预训练模型复用的各利益相关方提出建议。