Multi-Object Tracking (MOT) is evolving from geometric localization to Semantic MOT (SMOT) to answer complex relational queries, yet progress is hindered by semantic data scarcity and a structural disconnect between tracking architectures and Multi-modal Large Language Models (MLLMs). To address this, we introduce Grand-SMOT, a large-scale, open-world benchmark providing high-density, dual-stream narratives that comprehensively decouple individual behaviors from environmental contexts. Furthermore, we propose LLMTrack, the first framework to seamlessly integrate MLLMs into the SMOT task. LLMTrack establishes a Macro-Understanding-First paradigm, utilizing a novel Spatio-Temporal Fusion Module to align discrete geometric trajectories with continuous semantic features, effectively suppressing temporal hallucinations during online processing. Extensive experiments demonstrate that LLMTrack achieves state-of-the-art geometric tracking performance while delivering a qualitative leap in dynamic semantic reasoning. Notably, our analysis reveals that high-quality semantic narratives empower the language model to deduce complex social interactions naturally, demonstrating that direct cognitive reasoning is more effective than cumbersome explicit visual modeling. Ultimately, our contributions bridge the gap between perceptual tracking and cognitive reasoning, establishing a robust new foundation for comprehensive video understanding and intelligent narrative generation.

翻译：多目标跟踪（MOT）正从几何定位向语义多目标跟踪（SMOT）演进，以回答复杂的关系查询，但语义数据稀缺以及跟踪架构与多模态大语言模型（MLLMs）之间的结构脱节阻碍了进展。为此，我们引入了Grand-SMOT，这是一个大规模、开放世界基准，提供高密度、双流叙事，将个体行为与环境背景全面解耦。此外，我们提出了LLMTrack，这是首个将MLLMs无缝集成到SMOT任务中的框架。LLMTrack建立了一种“宏观理解优先”范式，利用新颖的时空融合模块将离散的几何轨迹与连续的语义特征对齐，有效抑制在线处理过程中的时序幻觉。大量实验表明，LLMTrack在实现最先进几何跟踪性能的同时，在动态语义推理方面实现了质的飞跃。值得注意的是，我们的分析表明，高质量的语义叙事能够使语言模型自然地推断复杂的社会交互，这证明直接的认知推理比繁琐的显式视觉建模更为有效。最终，我们的贡献弥合了感知跟踪与认知推理之间的鸿沟，为全面的视频理解和智能叙事生成奠定了坚实的新基础。