We introduce UbiPhysio, a milestone framework that delivers fine-grained action description and feedback in natural language to support people's daily functioning, fitness, and rehabilitation activities. This expert-like capability assists users in properly executing actions and maintaining engagement in remote fitness and rehabilitation programs. Specifically, the proposed UbiPhysio framework comprises a fine-grained action descriptor and a knowledge retrieval-enhanced feedback module. The action descriptor translates action data, represented by a set of biomechanical movement features we designed based on clinical priors, into textual descriptions of action types and potential movement patterns. Building on physiotherapeutic domain knowledge, the feedback module provides clear and engaging expert feedback. We evaluated UbiPhysio's performance through extensive experiments with data from 104 diverse participants, collected in a home-like setting during 25 types of everyday activities and exercises. We assessed the quality of the language output under different tuning strategies using standard benchmarks. We conducted a user study to gather insights from clinical physiotherapists and potential users about our framework. Our initial tests show promise for deploying UbiPhysio in real-life settings without specialized devices.

翻译：我们提出UbiPhysio，这是一个里程碑式的框架，能够以自然语言提供细粒度的动作描述与反馈，从而支持人们的日常功能活动、健身及康复训练。这种类专家的能力可帮助用户正确执行动作，并在远程健身与康复计划中保持参与度。具体而言，所提出的UbiPhysio框架包含一个细粒度动作描述器和一个增强知识检索的反馈模块。动作描述器将基于临床先验设计的一组生物力学运动特征所表征的动作数据，转化为动作类型与潜在运动模式的文本描述。反馈模块基于物理治疗领域知识，提供清晰且富有吸引力的专家级反馈。我们通过大量实验评估了UbiPhysio的性能，实验数据来自104名不同参与者在类似家庭环境中进行25种日常活动与运动时采集的数据。我们采用标准基准测试，评估了不同调优策略下语言输出的质量。我们还开展了一项用户研究，收集了临床物理治疗师及潜在用户对本框架的见解。初步测试表明，UbiPhysio有望在无需专用设备的现实场景中部署。