Mid-air gestures in Extended Reality (XR) often cause fatigue and imprecision. Surface-based interactions offer improved accuracy and comfort, but current egocentric vision methods struggle due to hand tracking challenges and unreliable surface plane estimation. We introduce SurfaceXR, a sensor fusion approach combining headset-based hand tracking with smartwatch IMU data to enable robust inputs on everyday surfaces. Our insight is that these modalities are complementary: hand tracking provides 3D positional data while IMUs capture high-frequency motion. A 21-participant study validates SurfaceXR's effectiveness for touch tracking and 8-class gesture recognition, demonstrating significant improvements over single-modality approaches.

翻译：在扩展现实（XR）中，空中手势常引发疲劳和精度不足问题。基于表面的交互虽能提升准确性与舒适度，但现有第一人称视角视觉方法受限于手部追踪挑战与不可靠的表面平面估计。我们提出SurfaceXR——一种融合头戴式设备手部追踪与智能手表IMU数据的传感器融合方法，以实现日常表面的鲁棒输入。核心理念在于这些模态互为补充：手部追踪提供三维位置数据，而IMU捕捉高频运动。基于21名参与者的实验验证了SurfaceXR在触摸追踪与8类手势识别任务中的有效性，相比单一模态方法展现出显著性能提升。