Machine learning (ML) training and inference often process data sets far exceeding GPU memory capacity, forcing them to rely on PCIe for on-demand tensor transfers, causing critical transfer bottlenecks. Lossy compression has been proposed to relieve bottlenecks but introduces workload-dependent accuracy loss, making it complex or even prohibitive to use in existing ML deployments. We explore lossless compression as an alternative that avoids this deployment complexity. We identify where lossless compression can be integrated into ML pipelines while minimizing interference with GPU execution. Based on our findings, we introduce Invariant Bit Packing (IBP), a novel lossless compression algorithm designed to minimize data transfer time for ML. IBP identifies and eliminates invariant bits across groups of tensors, improving throughput through GPU-optimized decompression that leverages warp parallelism, low-overhead bit operations, and asynchronous PCIe transfers. We provide easy-to-use APIs, showcasing them by adding IBP support to GNN training, as well as DLRM and LLM inference frameworks. IBP achieves, on average, 74% faster GNN training, 180% faster DLRM embedding lookup, and 24% faster LLM inference.

翻译：机器学习（ML）训练和推理经常处理远超GPU内存容量的数据集，迫使其依赖PCIe进行按需张量传输，导致关键的传输瓶颈。虽然已有研究提出有损压缩来缓解瓶颈，但这会引入与工作负载相关的精度损失，使得在现有ML部署中难以甚至无法使用有损压缩。我们探索无损压缩作为替代方案，以避免这种部署复杂性。我们确定了在ML流水线中集成无损压缩的位置，同时最小化对GPU执行的干扰。基于此，我们提出不变比特打包（Invariant Bit Packing，IBP），一种旨在最小化ML数据传输时间的新型无损压缩算法。IBP识别并消除张量组间的不变比特，通过GPU优化的解压缩（利用warp并行性、低开销比特操作和异步PCIe传输）来提高吞吐量。我们提供易用的API，并通过为GNN训练以及DLRM和LLM推理框架增加IBP支持来展示其效果。平均而言，IBP使GNN训练速度提升74%，DLRM嵌入查找速度提升180%，LLM推理速度提升24%。