Contemporary approaches to data management are increasingly relying on unified analytics and AI platforms to foster collaboration, interoperability, seamless access to reliable data, and high performance. Data Lakes featuring open standard table formats such as Delta Lake, Apache Hudi, and Apache Iceberg are central components of these data architectures. Choosing the right format for managing a table is crucial for achieving the objectives mentioned above. The challenge lies in selecting the best format, a task that is onerous and can yield temporary results, as the ideal choice may shift over time with data growth, evolving workloads, and the competitive development of table formats and processing engines. Moreover, restricting data access to a single format can hinder data sharing resulting in diminished business value over the long term. The ability to seamlessly interoperate between formats and with negligible overhead can effectively address these challenges. Our solution in this direction is an innovative omni-directional translator, XTable, that facilitates writing data in one format and reading it in any format, thus achieving the desired format interoperability. In this work, we demonstrate the effectiveness of XTable through application scenarios inspired by real-world use cases.

翻译：现代数据管理方法日益依赖统一分析与人工智能平台，以促进协作、互操作性、对可靠数据的高效访问以及高性能。采用开放标准表格式（如Delta Lake、Apache Hudi和Apache Iceberg）的数据湖是这些数据架构的核心组件。选择合适的表格式管理表格对于实现上述目标至关重要。挑战在于如何选择最佳格式——这一任务不仅繁重，且可能仅产生临时效果，因为理想选择会随时间推移因数据增长、工作负载演变以及表格式与处理引擎的竞争性发展而变化。此外，将数据访问限制为单一格式会阻碍数据共享，长期来看会导致业务价值下降。实现在不同格式间以极低开销无缝互操作的能力能够有效应对这些挑战。我们提出的解决方案是一种创新的全向翻译器XTable，它支持以某种格式写入数据并以任意格式读取，从而实现所需的格式互操作性。本文通过受真实应用场景启发的示例，验证了XTable的有效性。