As Extended Reality (XR) systems increasingly map and understand the physical world, interacting with these blended representations remains challenging. The current push for "natural" inputs has its trade-offs: touch is limited by human reach and fatigue, while gaze often lacks the precision for fine interaction. To bridge this gap, we introduce World Mouse, a cross-reality cursor that reinterprets the familiar 2D desktop mouse for complex 3D scenes. The system is driven by two core mechanisms: within-object interaction, which uses surface normals for precise cursor placement, and between-object navigation, which leverages interpolation to traverse empty space. Unlike previous virtual-only approaches, World Mouse leverages semantic segmentation and mesh reconstruction to treat physical objects as interactive surfaces. Through a series of prototypes, including object manipulation and screen-to-world transitions, we illustrate how cross-reality cursors may enable seamless interactions across real and virtual environments.

翻译：随着扩展现实系统日益精确地映射和理解物理世界，与这些混合现实表征进行交互仍然面临挑战。当前对"自然"输入方式的追求存在权衡：触控交互受限于人类手臂活动范围和疲劳问题，而视线交互往往缺乏精细操作的精度。为弥合这一鸿沟，我们提出了世界鼠标——一种将熟悉的二维桌面鼠标重新应用于复杂三维场景的跨现实光标系统。该系统的核心机制包含两方面：对象内交互利用表面法线实现精确光标定位，对象间导航通过插值算法实现空间跨越。与以往仅限于虚拟环境的方法不同，世界鼠标通过语义分割和网格重建技术，将物理对象转化为可交互表面。我们开发了一系列原型系统（包括物体操控和屏幕到现实的过渡场景），展示了跨现实光标如何实现真实与虚拟环境间的无缝交互。