Using intelligent systems to perceive psychological and social behaviors, that is, the underlying affective, cognitive, and pathological states that are manifested through observable behaviors and social interactions, remains a challenge due to their complex, multifaceted, and personalized nature. Existing work tackling these dimensions through specialized datasets and single-task systems often miss opportunities for scalability, cross-task transfer, and broader generalization. To address this gap, we curate Human Behavior Atlas, a unified benchmark of diverse behavioral tasks designed to support the development of foundation models for understanding psychological and social behaviors. Human Behavior Atlas comprises over 100,000 samples spanning text, audio, and visual modalities, covering tasks on affective states, cognitive states, pathologies, and social processes. Our unification efforts can reduce redundancy and cost, enable training to scale efficiently across tasks, and enhance generalization of behavioral features across domains. On Human Behavior Atlas, we train three models: Omnisapiens-7B SFT, Omnisapiens-7B BAM, and Omnisapiens-7B RL. We show that training on Human Behavior Atlas enables models to consistently outperform existing multimodal LLMs across diverse behavioral tasks. Pretraining on Human Behavior Atlas also improves transfer to novel behavioral datasets; with the targeted use of behavioral descriptors yielding meaningful performance gains. The benchmark, models, and codes can be found at: https://github.com/MIT-MI/human_behavior_atlas.

翻译：利用智能系统感知心理与社会行为——即通过可观察行为与社会互动所表现出的潜在情感、认知及病理状态——因其复杂、多面且个性化的本质而仍具挑战性。现有研究通过专用数据集与单任务系统处理这些维度时，往往错失了可扩展性、跨任务迁移及更广泛泛化的机遇。为填补这一空白，我们构建了人类行为图谱——一个统一的行为任务基准，旨在支持理解心理与社会行为的基础模型开发。人类行为图谱包含超过10万个涵盖文本、音频与视觉模态的样本，覆盖情感状态、认知状态、病理特征及社会过程等任务。我们的统一化工作能够减少冗余与成本，实现跨任务的高效规模化训练，并增强行为特征在跨领域中的泛化能力。基于人类行为图谱，我们训练了三个模型：Omnisapiens-7B SFT、Omnisapiens-7B BAM 与 Omnisapiens-7B RL。实验表明，在人类行为图谱上的训练能使模型在多样化行为任务中持续超越现有多模态大语言模型。基于人类行为图谱的预训练还提升了对新行为数据集的迁移能力；通过针对性使用行为描述符可产生显著性能提升。基准数据、模型及代码可通过以下链接获取：https://github.com/MIT-MI/human_behavior_atlas。