Wearable human activity recognition (WHAR) models often suffer from performance degradation under real-world cross-user distribution shifts. Test-time adaptation (TTA) mitigates this degradation by adapting models online using unlabeled test streams, yet existing methods largely inherit assumptions from vision tasks and underexploit the inherent inter-window temporal structure in WHAR streams. In this paper, we revisit such temporal structure as a feature-conditioned inference signal rather than merely an output-space smoothing prior. We derive the insight that temporal continuity and observation-induced feature deviations provide complementary cues for determining when to preserve or release temporal inertia and where to route prediction refinement during likely transitions. Building upon this insight, we propose SIGHT, a lightweight and backpropagation-free TTA framework for WHAR, enabling real-time edge deployment. SIGHT estimates predictive surprise by comparing the current feature with a prototype-based expected state, and then uses the resulting feature deviation to guide geometry-aware transition routing based on prototype alignment and stream-level marginal habit tracking. Evaluations on real-world datasets confirm that SIGHT outperforms existing TTA baselines while reducing computational and memory costs.

翻译：可穿戴人体活动识别（WHAR）模型在现实场景中常因跨用户分布偏移而导致性能下降。测试时自适应（TTA）方法通过利用无标注测试流在线调整模型来缓解这种性能退化，但现有方法大多继承视觉任务的假设，未能充分挖掘WHAR数据流中固有的跨窗口时序结构。本文重新审视这种时序结构——将其视为基于特征条件的推理信号，而非仅作为输出空间的平滑先验。我们揭示了一个关键洞察：时序连续性特征与观测引起的特征偏差互为补充线索，可用于判断何时保留或释放时序惯性，以及在可能的状态转换阶段指引预测精度的调整方向。基于此洞察，我们提出SIGHT——一种轻量级且无需反向传播的WHAR测试时自适应框架，支持实时边缘部署。SIGHT通过将当前特征与基于原型的期望状态进行比较来估计预测惊异度，进而利用所得特征偏差，经原型对齐与流级边际习惯追踪的协同作用，引导几何感知的转换路由策略。在真实数据集上的评估证实，SIGHT在降低计算与存储开销的同时，性能超越了现有TTA基线方法。