The Myers-Briggs Type Indicator (MBTI) is one of the most influential personality theories reflecting individual differences in thinking, feeling, and behaving. MBTI personality detection has garnered considerable research interest and has evolved significantly over the years. However, this task tends to be overly optimistic, as it currently does not align well with the natural distribution of population personality traits. Specifically, (1) the self-reported labels in existing datasets result in incorrect labeling issues, and (2) the hard labels fail to capture the full range of population personality distributions. In this paper, we optimize the task by constructing MBTIBench, the first manually annotated high-quality MBTI personality detection dataset with soft labels, under the guidance of psychologists. As for the first challenge, MBTIBench effectively solves the incorrect labeling issues, which account for 29.58% of the data. As for the second challenge, we estimate soft labels by deriving the polarity tendency of samples. The obtained soft labels confirm that there are more people with non-extreme personality traits. Experimental results not only highlight the polarized predictions and biases in LLMs as key directions for future research, but also confirm that soft labels can provide more benefits to other psychological tasks than hard labels. The code and data are available at https://github.com/Personality-NLP/MbtiBench.

翻译：迈尔斯-布里格斯类型指标（MBTI）是反映个体思维、感受和行为差异最具影响力的人格理论之一。MBTI人格检测已引起广泛研究关注，并在近年取得显著进展。然而，该任务目前存在过度乐观倾向，因其未能充分契合人口人格特质的自然分布。具体而言：（1）现有数据集中的自我报告标签导致错误标注问题；（2）硬标签无法完整捕捉人口人格分布的连续谱系。本文通过构建MBTIBench——首个在心理学专家指导下人工标注的软标签高质量MBTI人格检测数据集——对该任务进行优化。针对第一项挑战，MBTIBench有效解决了占比29.58%的错误标注问题。针对第二项挑战，我们通过推导样本的极性倾向来估计软标签。所得软标签证实了非极端人格特质人群的普遍存在。实验结果不仅揭示了大型语言模型的极化预测与偏见是未来研究的关键方向，同时验证了软标签相较于硬标签能为其他心理学任务提供更多助益。代码与数据已发布于https://github.com/Personality-NLP/MbtiBench。