In this paper, we address the challenges in running enterprise experimentation with hierarchical entities and present the methodologies behind the implementation of the Enterprise Experimentation Platform (EEP) at LinkedIn, which plays a pivotal role in delivering an intelligent, scalable, and reliable experimentation experience to optimize performance across all LinkedIn's enterprise products. We start with an introduction to the hierarchical entity relationships of the enterprise products and how such complex entity structure poses challenges to experimentation. We then delve into the details of our solutions for EEP including taxonomy based design setup with multiple entities, analysis methodologies in the presence of hierarchical entities, and advanced variance reduction techniques, etc. Recognizing the hierarchical ramping patterns inherent in enterprise experiments, we also propose a two-level Sample Size Ratio Mismatch (SSRM) detection methodology. This approach addresses SSRM at both the randomization unit and analysis unit levels, bolstering the internal validity and trustworthiness of analysis results within EEP. In the end, we discuss implementations and examine the business impacts of EEP through practical examples.

翻译：本文探讨了在分层实体环境下进行企业级实验所面临的挑战，并阐述了LinkedIn企业级实验平台（EEP）实施背后的方法论。该平台在LinkedIn所有企业级产品的性能优化中发挥着关键作用，旨在提供智能化、可扩展且可靠的实验体验。我们首先介绍了企业产品的分层实体关系，以及这种复杂的实体结构如何给实验带来挑战。随后，我们深入探讨了EEP解决方案的具体细节，包括基于多实体分类的设计配置、分层实体环境下的分析方法，以及先进的方差缩减技术等。基于对企业实验固有的分层扩展模式的认识，我们还提出了一种双层样本量比率失配检测方法。该方法在随机化单元和分析单元两个层面处理样本量比率失配问题，从而增强了EEP分析结果的内部效度与可信度。最后，我们通过实际案例讨论了EEP的实施过程，并评估了其业务影响。