Sparse tensors are prevalent in many data-intensive applications, yet existing differentiable programming frameworks are tailored towards dense tensors. This presents a significant challenge for efficiently computing gradients through sparse tensor operations, as their irregular sparsity patterns can result in substantial memory and computational overheads. In this work, we introduce a novel framework that enables the efficient and automatic differentiation of sparse tensors, addressing this fundamental issue. Our experiments demonstrate the effectiveness of the proposed framework in terms of performance and scalability, outperforming state-of-the-art frameworks across a range of synthetic and real-world datasets. Our approach offers a promising direction for enabling efficient and scalable differentiable programming with sparse tensors, which has significant implications for numerous applications in machine learning, natural language processing, and scientific computing.

翻译：稀疏张量在许多数据密集型应用中普遍存在，但现有可微分编程框架主要针对稠密张量设计。这导致通过稀疏张量运算高效计算梯度面临重大挑战，因其不规则稀疏模式可能带来显著的内存与计算开销。本文提出一种新型框架，能够实现稀疏张量的高效自动微分，从而解决这一根本性问题。实验表明，该框架在性能与可扩展性方面具有显著优势，在合成数据集与真实世界数据集上均优于现有先进框架。我们的方法为支持稀疏张量的高效可扩展可微分编程开辟了有前景的方向，这对机器学习、自然语言处理及科学计算等众多应用领域具有重要意义。