Snowpark enables Data Engineering and AI/ML workloads to run directly within Snowflake by deploying a secure sandbox on virtual warehouse nodes. This Snowpark Execution Environment (SEE) allows users to execute arbitrary workloads in Python and other languages in a secure and performant manner. As adoption has grown, the diversity of workloads has introduced increasingly sophisticated needs for sandboxing. To address these evolving requirements, Snowpark transitioned its in-house sandboxing solution to gVisor, augmented with targeted optimizations. This paper describes both the functional and performance objectives that guided the upgrade, outlines the new sandbox architecture, and details the challenges encountered during the journey, along with the solutions developed to resolve them. Finally, we present case studies that highlight new features enabled by the upgraded architecture, demonstrating SEE's extensibility and flexibility in supporting the next generation of Snowpark workloads.

翻译：Snowpark通过虚拟仓库节点部署安全沙箱，使数据工程与AI/ML工作负载能够直接在Snowflake中运行。该Snowpark执行环境（SEE）允许用户以安全且高性能的方式执行Python及其他语言的任意工作负载。随着采用规模扩大，工作负载的多样性对沙箱技术提出了日益复杂的需求。为应对这些不断演进的要求，Snowpark将其内部沙箱解决方案升级为基于gVisor的架构，并辅以针对性优化。本文阐述了指导此次升级的功能性与性能目标，概述了新型沙箱架构，详细记录了演进过程中遇到的技术挑战及相应的解决方案。最后，我们通过案例研究展示升级架构所支持的新特性，证明SEE在支撑下一代Snowpark工作负载方面具备的可扩展性与灵活性。