Many emerging cyber-physical systems, such as autonomous vehicles and robots, rely heavily on artificial intelligence and machine learning algorithms to perform important system operations. Since these highly parallel applications are computationally intensive, they need to be accelerated by graphics processing units (GPUs) to meet stringent timing constraints. However, despite the wide adoption of GPUs, efficiently scheduling multiple GPU applications while providing rigorous real-time guarantees remains a challenge. In this paper, we propose RTGPU, which can schedule the execution of multiple GPU applications in real-time to meet hard deadlines. Each GPU application can have multiple CPU execution and memory copy segments, as well as GPU kernels. We start with a model to explicitly account for the CPU and memory copy segments of these applications. We then consider the GPU architecture in the development of a precise timing model for the GPU kernels and leverage a technique known as persistent threads to implement fine-grained kernel scheduling with improved performance through interleaved execution. Next, we propose a general method for scheduling parallel GPU applications in real time. Finally, to schedule multiple parallel GPU applications, we propose a practical real-time scheduling algorithm based on federated scheduling and grid search (for GPU kernel segments) with uniprocessor fixed priority scheduling (for multiple CPU and memory copy segments). Our approach provides superior schedulability compared with previous work, and gives real-time guarantees to meet hard deadlines for multiple GPU applications according to comprehensive validation and evaluation on a real NVIDIA GTX1080Ti GPU system.

翻译：随着自动驾驶汽车和机器人等新兴信息物理系统的兴起，它们高度依赖人工智能与机器学习算法来执行关键系统操作。鉴于这些高度并行的应用计算密集度极高，必须借助图形处理器（GPU）加速以满足严格的时间约束。然而，尽管GPU已广泛普及，如何在提供严格实时保障的同时高效调度多个GPU应用仍是一大挑战。本文提出RTGPU系统，可实时调度多个GPU应用的执行以满足硬截止期限要求。每个GPU应用可包含多个CPU执行段、内存拷贝段及GPU内核。我们首先建立模型显式考虑这些应用的CPU执行段与内存拷贝段，继而结合GPU架构开发精确的GPU内核时序模型，并采用持久线程技术通过交错执行实现细粒度内核调度以提升性能。随后，我们提出通用的并行GPU应用实时调度方法。最后，针对多并行GPU应用的调度，我们提出基于联合调度与网格搜索（处理GPU内核段）结合单处理器固定优先级调度（处理多CPU与内存拷贝段）的实用实时调度算法。与现有工作相比，本方法具有更优的可调度性，并在真实NVIDIA GTX1080Ti GPU系统上的全面验证与评估中证明，能够为多个GPU应用提供满足硬截止期限的实时保障。