In this paper, we contribute to the debate on generative artificial intelligence (GenAI) in scientometrics. We argue that moving from a trial-and-error approach to an explainable and actionable use requires a principled understanding of strengths and weaknesses of GenAI as compared with other techniques and with human judgment. To this end, we introduce a conceptual framework based on the distinction between the semantic dimensions of texts, i.e. the meanings attributed to words, and their pragmatic dimension, i.e. their embedding within communicative situations. We leverage this framework to interpret the results of applications of GenAI in scientometrics and to provide guidance to users. Specifically, we conclude that key parameters to be considered are the nature of the task, the level of granularity of the analysis and whether the goal was descriptive, inferential or evaluative. These parameters lead to different strategies for using GenAI and human-machine integration. Finally, we suggest that, by generating large amounts of scientific language, GenAI might affect textual characteristics used to measure science, such as authors, words, and references. We argue that careful empirical work and theoretical reflection will be essential to remain capable of interpreting the evolving patterns of knowledge production in the age of AI.

翻译：本文旨在探讨生成式人工智能（GenAI）在科学计量学中的应用。我们认为，要从试错法转向可解释且可操作的应用，必须从根本上理解GenAI相较于其他技术及人类判断的优劣。为此，我们引入了一个基于文本语义维度（即词语所承载的意义）与其语用维度（即在交际情境中的嵌入方式）区分的概念框架。利用该框架，我们解读了GenAI在科学计量学中的应用结果，并为用户提供指导。具体而言，我们总结出需考虑的关键参数包括任务性质、分析粒度层级以及目标（描述性、推理性或评估性）。这些参数决定了使用GenAI及人机融合的不同策略。最后，我们指出，GenAI通过生成大量科学语言，可能影响用于衡量科学成果的文本特征（如作者、词语和参考文献）。我们认为，若要持续解读AI时代知识生产模式的演变特征，严谨的实证研究与理论反思至关重要。