In the traditional vehicular network, computing tasks generated by the vehicles are usually uploaded to the cloud for processing. However, since task offloading toward the cloud will cause a large delay, vehicular edge computing (VEC) is introduced to avoid such a problem and improve the whole system performance, where a roadside unit (RSU) with certain computing capability is used to process the data of vehicles as an edge entity. Owing to the privacy and security issues, vehicles are reluctant to upload local data directly to the RSU, and thus federated learning (FL) becomes a promising technology for some machine learning tasks in VEC, where vehicles only need to upload the local model hyperparameters instead of transferring their local data to the nearby RSU. Furthermore, as vehicles have different local training time due to various sizes of local data and their different computing capabilities, asynchronous federated learning (AFL) is employed to facilitate the RSU to update the global model immediately after receiving a local model to reduce the aggregation delay. However, in AFL of VEC, different vehicles may have different impact on the global model updating because of their various local training delay, transmission delay and local data sizes. Also, if there are bad nodes among the vehicles, it will affect the global aggregation quality at the RSU. To solve the above problem, we shall propose a deep reinforcement learning (DRL) based vehicle selection scheme to improve the accuracy of the global model in AFL of vehicular network. In the scheme, we present the model including the state, action and reward in the DRL based to the specific problem. Simulation results demonstrate our scheme can effectively remove the bad nodes and improve the aggregation accuracy of the global model.

翻译：在传统车辆网络中，车辆产生的计算任务通常上传至云端处理。然而，由于任务卸载至云端会带来较大延迟，引入车联网边缘计算（VEC）以避免该问题并提升系统整体性能，其中具有计算能力的路侧单元（RSU）作为边缘节点处理车辆数据。考虑到隐私与安全问题，车辆不愿直接向RSU上传本地数据，因此联邦学习（FL）成为VEC中机器学习任务的重要技术——车辆仅需上传本地模型超参数，而非将本地数据传输至附近RSU。此外，由于车辆本地数据规模及计算能力差异导致训练时长不同，采用异步联邦学习（AFL）可使RSU在接收到任一本地模型后立即更新全局模型，从而减少聚合延迟。然而，在VEC的AFL框架中，不同车辆因本地训练延迟、传输延迟及数据规模差异，对全局模型更新的影响程度各异。同时，若车辆中存在恶意节点，将影响RSU的全局聚合质量。针对上述问题，本文提出基于深度强化学习（DRL）的车辆选择方案，以提升车辆网络AFL中全局模型的准确性。该方案针对具体问题设计了包含状态、动作和奖励的DRL模型。仿真结果表明，该方案能有效剔除恶意节点，改善全局模型的聚合精度。