This study investigates the use of large language models (LLMs) for human behavior understanding by jointly leveraging motion and video data. We argue that integrating these complementary modalities is essential for capturing both fine-grained motion dynamics and contextual semantics of human actions, addressing the limitations of prior motion-only or video-only approaches. To this end, we propose ViMoNet, a multimodal vision-language framework trained through a two-stage alignment and instruction-tuning strategy that combines precise motion-text supervision with large-scale video-text data. We further introduce VIMOS, a multimodal dataset comprising human motion sequences, videos, and instruction-level annotations, along with ViMoNet-Bench, a standardized benchmark for evaluating behavior-centric reasoning. Experimental results demonstrate that ViMoNet consistently outperforms existing methods across caption generation, motion understanding, and human behavior interpretation tasks. The proposed framework shows significant potential in assistive healthcare applications, such as elderly monitoring, fall detection, and early identification of health risks in aging populations. This work contributes to the United Nations Sustainable Development Goal 3 (SDG 3: Good Health and Well-being) by enabling accessible AI-driven tools that promote universal health coverage, reduce preventable health issues, and enhance overall well-being.

翻译：本研究探讨了利用大型语言模型（LLMs）通过联合利用运动和视频数据来理解人类行为。我们认为，整合这两种互补模态对于捕捉细粒度的运动动态和人类动作的上下文语义至关重要，从而解决了先前仅使用运动或仅使用视频方法的局限性。为此，我们提出了ViMoNet，这是一个通过两阶段对齐与指令微调策略训练的多模态视觉语言框架，该策略将精确的运动-文本监督与大规模视频-文本数据相结合。我们进一步引入了VIMOS，一个包含人体运动序列、视频和指令级标注的多模态数据集，以及用于评估以行为为中心推理的标准化基准ViMoNet-Bench。实验结果表明，ViMoNet在描述生成、运动理解和人类行为解释任务上均持续优于现有方法。所提出的框架在辅助医疗应用中显示出巨大潜力，例如老年人监护、跌倒检测以及老年人群健康风险的早期识别。这项工作通过开发可访问的AI驱动工具，促进全民健康覆盖、减少可预防的健康问题并提升整体福祉，从而为联合国可持续发展目标3（SDG 3：良好健康与福祉）做出了贡献。