Moral framing and sentiment can affect a variety of online and offline behaviors, including donation, environmental action, political engagement, and protest. Various computational methods in Natural Language Processing (NLP) have been used to detect moral sentiment from textual data, but achieving strong performance in such subjective tasks requires large, hand-annotated datasets. Previous corpora annotated for moral sentiment have proven valuable, and have generated new insights both within NLP and across the social sciences, but have been limited to Twitter. To facilitate improving our understanding of the role of moral rhetoric, we present the Moral Foundations Reddit Corpus, a collection of 16,123 English Reddit comments that have been curated from 12 distinct subreddits, hand-annotated by at least three trained annotators for 8 categories of moral sentiment (i.e., Care, Proportionality, Equality, Purity, Authority, Loyalty, Thin Morality, Implicit/Explicit Morality) based on the updated Moral Foundations Theory (MFT) framework. We evaluate baselines using large language models (Llama3-8B, Ministral-8B) in zero-shot, few-shot, and PEFT (Parameter-Efficient Fine-Tuning) settings, comparing their performance to fine-tuned encoder-only models like BERT (Bidirectional Encoder Representations from Transformers). The results show that LLMs continue to lag behind fine-tuned encoders on this subjective task, underscoring the ongoing need for human-annotated moral corpora for AI alignment evaluation. Keywords: moral sentiment annotation, moral values, moral foundations theory, multi-label text classification, large language models, benchmark dataset, evaluation and alignment resource

翻译：道德框架和情感能够影响多种线上及线下行为，包括捐款、环保行动、政治参与和抗议。自然语言处理（NLP）领域的多种计算方法已被用于从文本数据中检测道德情感，但在此类主观任务中取得优异性能需要大量人工标注的数据集。此前标注道德情感的语料库已被证明具有重要价值，并在NLP及社会科学领域产生了新的洞见，但这些语料库仅局限于Twitter平台。为促进对道德修辞作用的理解，我们提出了道德基础Reddit语料库（Moral Foundations Reddit Corpus），该语料库包含从12个不同子版块筛选出的16,123条英语Reddit评论，由至少三名训练有素的标注者依据更新后的道德基础理论（MFT）框架，对8类道德情感（即关爱、公平、平等、纯洁、权威、忠诚、薄道德、隐式/显式道德）进行手工标注。我们基于大语言模型（Llama3-8B、Ministral-8B）在零样本、少样本及参数高效微调（PEFT）设置下评估了基线模型，并将其性能与微调后的仅编码器模型（如BERT）进行了对比。结果表明，在此主观任务上，大语言模型仍落后于微调后的编码器模型，凸显了人工标注道德语料库在人工智能对齐评估中的持续需求。关键词：道德情感标注，道德价值观，道德基础理论，多标签文本分类，大语言模型，基准数据集，评估与对齐资源