The emergence of 5G networks has enabled the deployment of a two-tier edge and vehicular-fog network. It comprises Multi-access Edge Computing (MEC) and Vehicular-Fogs (VFs), strategically positioned closer to Internet of Things (IoT) devices, reducing propagation latency compared to cloud-based solutions and ensuring satisfactory quality of service (QoS). However, during high-traffic events like concerts or athletic contests, MEC sites may face congestion and become overloaded. Utilizing offloading techniques, we can transfer computationally intensive tasks from resource-constrained devices to those with sufficient capacity, for accelerating tasks and extending device battery life. In this research, we consider offloading within a two-tier MEC and VF architecture, involving offloading from MEC to MEC and from MEC to VF. The primary objective is to minimize the average system cost, considering both latency and energy consumption. To achieve this goal, we formulate a multi-objective optimization problem aimed at minimizing latency and energy while considering given resource constraints. To facilitate decision-making for nearly optimal computational offloading, we design an equivalent reinforcement learning environment that accurately represents the network architecture and the formulated problem. To accomplish this, we propose a Distributed-TD3 (DTD3) approach, which builds on the TD3 algorithm. Extensive simulations, demonstrate that our strategy achieves faster convergence and higher efficiency compared to other benchmark solutions.

翻译：5G网络的出现使得双层边缘与车辆雾网络得以部署。该网络由多接入边缘计算（MEC）和车辆雾（VFs）构成，战略性地部署在更靠近物联网（IoT）设备的位置，相较于基于云的解决方案可降低传播延迟，并确保令人满意的服务质量（QoS）。然而，在音乐会或体育赛事等高流量事件中，MEC站点可能出现拥塞和过载。利用卸载技术，我们可以将计算密集型任务从资源受限设备转移到容量充足的设备，以加速任务执行并延长设备电池寿命。在本研究中，我们考虑双层MEC与VF架构中的卸载问题，涉及MEC到MEC以及MEC到VF的卸载。主要目标是在考虑延迟和能耗的情况下，最小化平均系统成本。为实现此目标，我们构建了一个多目标优化问题，旨在给定资源约束下最小化延迟与能耗。为支持接近最优计算卸载的决策制定，我们设计了一个等效的强化学习环境，该环境准确表征网络架构与所构建问题。为此，我们提出一种基于TD3算法的分布式TD3（DTD3）方法。大量仿真实验表明，与其他基准解决方案相比，我们的策略实现了更快的收敛速度和更高的效率。