Social interaction depends on both language and visible social signals, such as facial expressions, posture, gaze, and emotional shifts. Yet existing social-agent benchmarks are largely text-based and rarely test whether multimodal agents can use visual cues to guide interaction. We introduce \textsc{\benchmarkname{}}, a benchmark evaluating visual social intelligence in multimodal social simulation. It contains 240 scenarios, 585 role instances, and 2,340 role-task instances, combining aligned textual-visual evidence, structured role profiles, and four role-level tasks: expression task, characteristic task, interaction regulation task, and interaction outcome task. Evaluating seven recent MLLMs under verbalized-vision and direct-vision reveals a clear gap between local role enactment and interaction management: role-specific expression and conflict handling are near saturation, whereas interaction regulation and visually grounded outcome achievement remain substantially more difficult. The code is released at https://github.com/JunsWan/AgentViSS, and the dataset is available at https://huggingface.co/datasets/JunsWan/AgentViSS.

翻译：社交互动既依赖于语言，也依赖于可见的社会信号，例如面部表情、姿势、注视和情绪变化。然而，现有的社交智能体基准测试大多基于文本，很少检验多模态智能体是否能利用视觉线索来引导互动。我们引入了\textsc{\benchmarkname{}}，一个评估多模态社会模拟中视觉社会智能的基准测试。它包含240个场景、585个角色实例和2,340个角色任务实例，结合了对齐的文本-视觉证据、结构化的角色档案，以及四个角色级任务：表情任务、特征任务、互动调节任务和互动结果任务。在口头视觉和直接视觉模式下评估了七个最新的多模态大语言模型，揭示了局部角色扮演与互动管理之间存在明显差距：特定角色的表情和冲突处理已接近饱和，而互动调节和基于视觉的结果达成仍然困难得多。代码发布于https://github.com/JunsWan/AgentViSS，数据集发布于https://huggingface.co/datasets/JunsWan/AgentViSS。