This demonstration presents a new Open Source SQL-to-SQL compiler for Incremental View Maintenance (IVM). While previous systems, such as DBToaster, implemented computational functionality for IVM in a separate system, the core principle of OpenIVM is to make use of existing SQL query processing engines and perform all IVM computations via SQL. This approach enables the integration of IVM in these systems without code duplication. Also, it eases its use in cross-system IVM, i.e. to orchestrate an HTAP system in which one (OLTP) DBMS provides insertions/updates/deletes (deltas), which are propagated using SQL into another (OLAP) DBMS, hosting materialized views. Our system compiles view definitions into SQL to eventually propagate deltas into the table that materializes the view, following the principles of DBSP. Under the hood, OpenIVM uses the DuckDB library to compile (parse, transform, optimize) the materialized view maintenance logic. We demonstrate OpenIVM in action (i) as the core of a DuckDB extension module that adds IVM functionality to it and (ii) powering cross-system IVM for HTAP, with PostgreSQL handling updates on base tables and DuckDB hosting materialized views on these.
翻译:本演示介绍了一种新的开源SQL到SQL编译器,用于增量视图维护(IVM)。与DBToaster等先前系统将IVM计算功能实现在独立系统中不同,OpenIVM的核心原则是利用现有SQL查询处理引擎,通过SQL执行所有IVM计算。该方法无需代码重复即可将IVM集成到这些系统中,同时便于跨系统IVM的应用,即协调HTAP系统,使一个(OLTP)数据库管理系统提供插入/更新/删除(变更数据),并通过SQL将这些变更传播到另一个托管物化视图的(OLAP)数据库管理系统。我们的系统遵循DBSP原理,将视图定义编译为SQL,最终将变更数据传播至物化视图的表中。在底层,OpenIVM使用DuckDB库来编译(解析、变换、优化)物化视图维护逻辑。我们展示了OpenIVM的实际应用:(i)作为DuckDB扩展模块的核心,为其添加IVM功能;(ii)驱动用于HTAP的跨系统IVM,由PostgreSQL处理基表更新,DuckDB托管这些基表的物化视图。