Mixed Reality (MR)-aided operation overlays digital objects on the physical world to provide a more immersive and intuitive operation process. A primary challenge is the precise and fast auto-verification of whether the user follows MR guidance by comparing frames before and after each operation. The pre-operation frame includes virtual guiding objects, while the post-operation frame contains physical counterparts. Existing approaches fall short of accounting for the discrepancies between physical and virtual objects due to imperfect 3D modeling or lighting estimation. In this paper, we propose EVER: an edge-assisted auto-verification system for mobile MR-aided operations. Unlike traditional frame-based similarity comparisons, EVER leverages the segmentation model and rendering pipeline adapted to the unique attributes of frames with physical pieces and those with their virtual counterparts; it adopts a threshold-based strategy using Intersection over Union (IoU) metrics for accurate auto-verification. To ensure fast auto-verification and low energy consumption, EVER offloads compute-intensive tasks to an edge server. Through comprehensive evaluations of public datasets and custom datasets with practical implementation, EVER achieves over 90% verification accuracy within 100 milliseconds (significantly faster than average human reaction time of approximately 273 milliseconds), while consuming only minimal additional computational resources and energy compared to a system without auto-verification.

翻译：混合现实（MR）辅助操作通过将数字对象叠加于物理世界，提供更具沉浸感与直觉性的操作流程。其核心挑战在于通过比较每次操作前后的帧图像，实现精准快速的自动验证，以判断用户是否遵循MR引导。操作前帧包含虚拟引导对象，而操作后帧则包含对应的物理实体。现有方法因未能充分考虑三维建模或光照估计不完善导致的物理与虚拟对象间差异，存在明显不足。本文提出EVER：一种面向移动MR辅助操作的边缘辅助自动验证系统。与传统基于帧相似度比较的方法不同，EVER利用针对含物理部件帧与含虚拟对应物帧独特属性优化的分割模型与渲染管线；采用基于交并比（IoU）指标的阈值策略实现精准自动验证。为确保快速验证与低能耗，EVER将计算密集型任务卸载至边缘服务器。通过对公开数据集及实际构建的自定义数据集进行全面评估，EVER在100毫秒内（显著快于人类平均反应时间约273毫秒）达到超过90%的验证准确率，同时相较于无自动验证系统，仅消耗极少量额外计算资源与能量。