Well curated, large-scale corpora of social media posts containing broad public opinion offer an alternative data source to complement traditional surveys. While surveys are effective at collecting representative samples and are capable of achieving high accuracy, they can be both expensive to run and lag public opinion by days or weeks. Both of these drawbacks could be overcome with a real-time, high volume data stream and fast analysis pipeline. A central challenge in orchestrating such a data pipeline is devising an effective method for rapidly selecting the best corpus of relevant documents for analysis. Querying with keywords alone often includes irrelevant documents that are not easily disambiguated with bag-of-words natural language processing methods. Here, we explore methods of corpus curation to filter irrelevant tweets using pre-trained transformer-based models, fine-tuned for our binary classification task on hand-labeled tweets. We are able to achieve F1 scores of up to 0.95. The low cost and high performance of fine-tuning such a model suggests that our approach could be of broad benefit as a pre-processing step for social media datasets with uncertain corpus boundaries.

翻译：精心策展的大规模社交媒体帖子语料库，其中包含广泛的公众舆论，为补充传统调查提供了替代数据源。尽管调查在收集代表性样本方面效果显著且能达到高准确率，但其运行成本高昂，且结果滞后于公众舆论数天或数周。这两个缺陷可通过实时、高容量数据流与快速分析管线加以克服。构建此类数据管线的核心挑战在于设计一种高效方法，以快速选择最相关的文档语料库进行分析。仅使用关键词查询往往会包含难以通过词袋自然语言处理方法消除歧义的非相关文档。本研究探索了基于预训练Transformer模型（针对手工标注推文的二分类任务进行微调）的语料库策展方法，以过滤无关推文。我们实现了高达0.95的F1分数。微调此类模型的低成本与高性能表明，该方法作为边界不确定社交媒体数据集的预处理步骤，具有广泛应用价值。