Previous studies have highlighted the importance of vaccination as an effective strategy to control the transmission of the COVID-19 virus. It is crucial for policymakers to have a comprehensive understanding of the public's stance towards vaccination on a large scale. However, attitudes towards COVID-19 vaccination, such as pro-vaccine or vaccine hesitancy, have evolved over time on social media. Thus, it is necessary to account for possible temporal shifts when analysing these stances. This study aims to examine the impact of temporal concept drift on stance detection towards COVID-19 vaccination on Twitter. To this end, we evaluate a range of transformer-based models using chronological (split the training, validation and testing sets in the order of time) and random splits (randomly split these three sets) of social media data. Our findings demonstrate significant discrepancies in model performance when comparing random and chronological splits across all monolingual and multilingual datasets. Chronological splits significantly reduce the accuracy of stance classification. Therefore, real-world stance detection approaches need to be further refined to incorporate temporal factors as a key consideration.

翻译：既往研究已强调疫苗接种作为控制COVID-19病毒传播有效策略的重要性。政策制定者亟需大规模全面了解公众对疫苗接种的立场。然而，社交媒体上针对COVID-19疫苗接种的态度（如支持接种或疫苗犹豫）随时间演变。因此，在分析这些立场时，有必要考虑潜在的时间动态变化。本研究旨在探究时间概念漂移对Twitter平台上COVID-19疫苗接种立场检测的影响。为此，我们采用时间顺序划分（按时间顺序分割训练集、验证集和测试集）与随机划分（随机分割这三个集合）两种方式处理社交媒体数据，评估了多种基于Transformer的模型。研究结果表明，在所有单语和多语言数据集中，随机划分与时间顺序划分下的模型性能存在显著差异。时间顺序划分显著降低了立场分类的准确性。因此，实际场景中的立场检测方法需进一步优化，将时间因素作为关键考量纳入其中。