This study focuses on media bias detection, crucial in today's era of influential social media platforms shaping individual attitudes and opinions. In contrast to prior work that primarily relies on training specific models tailored to particular datasets, resulting in limited adaptability and subpar performance on out-of-domain data, we introduce a general bias detection framework, IndiVec, built upon large language models. IndiVec begins by constructing a fine-grained media bias database, leveraging the robust instruction-following capabilities of large language models and vector database techniques. When confronted with new input for bias detection, our framework automatically selects the most relevant indicator from the vector database and employs majority voting to determine the input's bias label. IndiVec excels compared to previous methods due to its adaptability (demonstrating consistent performance across diverse datasets from various sources) and explainability (providing explicit top-k indicators to interpret bias predictions). Experimental results on four political bias datasets highlight IndiVec's significant superiority over baselines. Furthermore, additional experiments and analysis provide profound insights into the framework's effectiveness.
翻译:本研究聚焦于媒体偏见检测,这在当今社交媒体平台对个人态度和观点具有显著影响力的时代至关重要。与先前主要依赖针对特定数据集训练专用模型、导致适应能力有限且在域外数据上表现欠佳的工作不同,我们提出了一种基于大语言模型的通用偏见检测框架IndiVec。IndiVec首先通过构建细粒度媒体偏见数据库,并利用大语言模型强大的指令遵循能力以及向量数据库技术实现。当面对新的输入进行偏见检测时,该框架自动从向量数据库中选取最相关的指标,并采用多数投票机制确定输入的偏见标签。相较先前方法,IndiVec在适应性(在来自不同来源的多个数据集上表现稳定)和可解释性(提供明确的top-k指标以解释偏见预测)方面表现卓越。在四个政治偏见数据集上的实验结果表明,IndiVec显著优于基线方法。此外,进一步的实验与分析为该框架的有效性提供了深刻洞见。