Real-time Digital Twinning of physical world scenes onto the Metaverse is necessary for a myriad of applications such as augmented-reality (AR) assisted driving. In AR assisted driving, physical environment scenes are first captured by Internet of Vehicles (IoVs) and are uploaded to the Metaverse. A central Metaverse Map Service Provider (MMSP) will aggregate information from all IoVs to develop a central Metaverse Map. Information from the Metaverse Map can then be downloaded into individual IoVs on demand and be delivered as AR scenes to the driver. However, the growing interest in developing AR assisted driving applications which relies on digital twinning invites adversaries. These adversaries may place physical adversarial patches on physical world objects such as cars, signboards, or on roads, seeking to contort the virtual world digital twin. Hence, there is a need to detect these physical world adversarial patches. Nevertheless, as real-time, accurate detection of adversarial patches is compute-intensive, these physical world scenes have to be offloaded to the Metaverse Map Base Stations (MMBS) for computation. Hence in our work, we considered an environment with moving Internet of Vehicles (IoV), uploading real-time physical world scenes to the MMBSs. We formulated a realistic joint variable optimization problem where the MMSPs' objective is to maximize adversarial patch detection mean average precision (mAP), while minimizing the computed AR scene up-link transmission latency and IoVs' up-link transmission idle count, through optimizing the IoV-MMBS allocation and IoV up-link scene resolution selection. We proposed a Heterogeneous Action Proximal Policy Optimization (HAPPO) (discrete-continuous) algorithm to tackle the proposed problem. Extensive experiments shows HAPPO outperforms baseline models when compared against key metrics.

翻译：将物理世界场景实时数字孪生至元宇宙是增强现实辅助驾驶等诸多应用的必要条件。在增强现实辅助驾驶中，物理环境场景首先由车联网设备捕获并上传至元宇宙。中央元宇宙地图服务提供商将整合所有车联网的信息以构建中央元宇宙地图。单个车联网可按需下载元宇宙地图信息，并以增强现实场景形式呈现给驾驶员。然而，基于数字孪生的增强现实辅助驾驶应用日益增长的需求也引来了攻击者。这些攻击者可能在汽车、路牌或道路等物理世界物体上放置物理对抗补丁，试图扭曲虚拟世界的数字孪生。因此，需要检测这些物理世界对抗补丁。然而，由于实时、准确地检测对抗补丁属于计算密集型任务，这些物理世界场景必须卸载至元宇宙地图基站进行计算。因此，在我们的工作中，考虑了移动车联网实时上传物理世界场景至元宇宙地图基站的环境。我们构建了一个现实的联合变量优化问题，其中元宇宙地图服务提供商的目标是通过优化车联网-基站分配和车联网上行场景分辨率选择，在最大化对抗补丁检测平均精度的同时，最小化计算出的增强现实场景上行传输延迟和车联网上行传输空闲计数。我们提出了一种异构动作近端策略优化（离散-连续）算法来解决该问题。大量实验表明，与基线模型相比，我们的算法在关键指标上表现更优。