The challenges associated with using pre-trained models (PTMs) have not been specifically investigated, which hampers their effective utilization. To address this knowledge gap, we collected and analyzed a dataset of 5,896 PTM-related questions on Stack Overflow. We first analyze the popularity and difficulty trends of PTM-related questions. We find that PTM-related questions are becoming more and more popular over time. However, it is noteworthy that PTM-related questions not only have a lower response rate but also exhibit a longer response time compared to many well-researched topics in software engineering. This observation emphasizes the significant difficulty and complexity associated with the practical application of PTMs. To delve into the specific challenges, we manually annotate 430 PTM-related questions, categorizing them into a hierarchical taxonomy of 42 codes (i.e., leaf nodes) and three categories. This taxonomy encompasses many PTM prominent challenges such as fine-tuning, output understanding, and prompt customization, which reflects the gaps between current techniques and practical needs. We discuss the implications of our study for PTM practitioners, vendors, and educators, and suggest possible directions and solutions for future research.

翻译：预训练模型（PTM）使用过程中的挑战尚未得到专门研究，这阻碍了其有效应用。为填补这一知识空白，我们收集并分析了Stack Overflow上5,896个与PTM相关的问题。首先分析了PTM相关问题的流行度与难度趋势，发现此类问题随时间推移日益增多。值得注意的是，与软件工程中许多深入研究过的主题相比，PTM相关问题不仅回复率较低，且响应时间更长。这一观察结果凸显了PTM实际应用中的显著困难与复杂性。为深入探究具体挑战，我们人工标注了430个PTM相关问题，将其归类为包含42个代码（即叶节点）和三个类别的层次化分类体系。该分类涵盖了微调、输出理解、提示定制等PTM领域的突出挑战，反映了当前技术与实际需求之间的差距。我们讨论了研究结果对PTM从业者、供应商及教育者的启示，并提出了未来研究可能的发展方向与解决方案。