The exponential growth of artificial intelligence (AI) and machine learning (ML) applications has necessitated the development of efficient storage solutions for vector and tensor data. This paper presents a novel approach for tensor storage in a Lakehouse architecture using Delta Lake. By adopting the multidimensional array storage strategy from array databases and sparse encoding methods to Delta Lake tables, experiments show that this approach has demonstrated notable improvements in both space and time efficiencies when compared to traditional serialization of tensors. These results provide valuable insights for the development and implementation of optimized vector and tensor storage solutions in data-intensive applications, contributing to the evolution of efficient data management practices in AI and ML domains in cloud-native environments

翻译：人工智能与机器学习应用的指数级增长，催生了对向量与张量数据高效存储解决方案的迫切需求。本文提出了一种基于Delta Lake的湖仓一体架构下张量存储的创新方法。通过将数组数据库中的多维数组存储策略与稀疏编码方法引入Delta Lake表，实验表明，相较于传统的张量序列化方式，该方法在空间效率与时间效率上均取得了显著提升。这些成果为数据密集型应用中优化的向量与张量存储解决方案的研发与实现提供了宝贵洞见，推动了云原生环境下人工智能与机器学习领域高效数据管理实践的演进。