Understanding human actions from visual observations is essential for human--robot interaction, particularly when semantic interpretation of unfamiliar or hard-to-annotate actions is required. In scenarios such as rapid and less common activities, collecting sufficient labeled data for supervised learning is challenging, making zero-shot approaches a practical alternative for semantic understanding without task-specific training. While recent advances in large-scale pretrained models enable such zero-shot reasoning, the impact of temporal resolution, especially for rapid and fine-grained motions, remains underexplored. In this study, we investigate how temporal resolution affects zero-shot semantic understanding of high-speed human actions. Using kendo as a representative case of rapid and subtle motion patterns, we propose a training-free pipeline that combines a pre-trained video-language model for semantic representation with large language model-based reasoning for pairwise action comparison. Through controlled experiments across multiple frame rates (120 Hz, 60 Hz, and 30 Hz), we show that higher temporal resolution significantly improves semantic separability in zero-shot settings. We further analyze the role of tracking-based human joint information under both full and partial observation scenarios. Quantitative evaluation using a nearest-class prototype strategy demonstrates that high-speed video provides more stable and interpretable semantic representations for fast actions. These findings highlight the importance of temporal resolution in training-free action recognition and suggest that high-speed perception can enhance semantic understanding capabilities.
翻译:从视觉观测中理解人类动作是人机交互的核心需求,尤其当需要语义解读不熟悉或难以标注的动作时。在快速及少见动作等场景中,收集充足标注数据进行监督学习面临挑战,这使得零样本方法成为无需任务特定训练即可实现语义理解的实用替代方案。尽管大规模预训练模型的最新进展已支持此类零样本推理,但时间分辨率(尤其对快速细微运动)的影响仍未得到充分探索。本研究探究时间分辨率如何影响高速人类动作的零样本语义理解。以剑道作为快速细微运动模式的典型案例,我们提出了一种无训练流水线,将用于语义表征的预训练视频-语言模型与基于大语言模型的成对动作比较推理相结合。通过多帧率(120Hz、60Hz、30Hz)控制实验表明,更高时间分辨率能显著提升零样本场景中的语义可分性。进一步分析了全观测与部分观测条件下基于追踪的人体关节信息的作用机制。采用最近类原型策略的定量评估显示,高速视频可为快速动作提供更稳定且可解释的语义表征。这些发现揭示了时间分辨率在无训练动作识别中的重要性,并表明高速感知能够增强语义理解能力。