NVMe(Non-Volatile Memory Express) is an industry standard for solid-state drives (SSDs) that has been widely adopted in data centers. NVMe virtualization is crucial in cloud computing as it allows for virtualized NVMe devices to be used by virtual machines (VMs), thereby improving the utilization of storage resources. However, traditional software-based solutions have flexibility benefits but often come at the cost of performance degradation or high CPU overhead. On the other hand, hardware-assisted solutions offer high performance and low CPU usage, but their adoption is often limited by the need for special hardware support or the requirement for new hardware development. In this paper, we propose LightIOV, a novel software-based NVMe virtualization mechanism that achieves high performance and scalability without consuming valuable CPU resources and without requiring special hardware support. LightIOV can support thousands of VMs on each server. The key idea behind LightIOV is NVMe hardware I/O queues passthrough, which enables VMs to directly access I/O queues of NVMe devices, thus eliminating virtualization overhead and providing near-native performance. Results from our experiments show that LightIOV can provide comparable performance to VFIO, with an IOPS of 97.6%-100.2% of VFIO. Furthermore, in high-density VMs environments, LightIOV achieves 31.4% lower latency than SPDK-Vhost when running 200 VMs, and an improvement of 27.1% in OPS performance in real-world applications.

翻译：NVMe（非易失性内存快速通道）是固态硬盘（SSD）的行业标准，已在数据中心得到广泛采用。NVMe虚拟化在云计算中至关重要，它允许虚拟机使用虚拟化的NVMe设备，从而提高存储资源的利用率。然而，传统的基于软件的解决方案虽具备灵活性优势，但常以性能下降或高CPU开销为代价。另一方面，硬件辅助方案可实现高性能和低CPU占用，但其应用常受限于对特殊硬件支持或新硬件开发的需求。本文提出LightIOV——一种新颖的基于软件的NVMe虚拟化机制，在不消耗宝贵CPU资源且无需特殊硬件支持的前提下，实现高性能与可扩展性。LightIOV可在每台服务器上支持数千个虚拟机。其核心思想是NVMe硬件I/O队列直通技术，使虚拟机能够直接访问NVMe设备的I/O队列，从而消除虚拟化开销并提供接近原生的性能。实验结果表明，LightIOV可提供与VFIO相当的性能，IOPS可达VFIO的97.6%-100.2%。此外，在虚拟机高密度环境中，运行200个虚拟机时，LightIOV的延迟比SPDK-Vhost低31.4%，在实际应用中的OPS性能提升了27.1%。