Making ideal decisions as a product leader in a web-facing company is extremely difficult. In addition to navigating the ambiguity of customer satisfaction and achieving business goals, one must also pave a path forward for ones' products and services to remain relevant, desirable, and profitable. Data and experimentation to test product hypotheses are key to informing product decisions. Online controlled experiments by A/B testing may provide the best data to support such decisions with high confidence, but can be time-consuming and expensive, especially when one wants to understand impact to key business metrics such as retention or long-term value. Offline experimentation allows one to rapidly iterate and test, but often cannot provide the same level of confidence, and cannot easily shine a light on impact on business metrics. We introduce a novel, lightweight, and flexible approach to investigating hypotheses, called scenario analysis, that aims to support product leaders' decisions using data about users and estimates of business metrics. Its strengths are that it can provide guidance on trade-offs that are incurred by growing or shifting consumption, estimate trends in long-term outcomes like retention and other important business metrics, and can generate hypotheses about relationships between metrics at scale.

翻译：作为面向网络公司的产品领导者，做出理想决策极其困难。除了要应对用户满意度与业务目标之间的模糊性，还必须为产品和服务规划发展路径，以确保其保持相关性、吸引力及盈利能力。用于测试产品假设的数据和实验是支撑产品决策的关键。通过A/B测试进行的在线受控实验或许能为决策提供高置信度的最佳数据支持，但此类实验耗时且成本高昂，尤其当需了解其对留存率或长期价值等关键业务指标的影响时。离线实验虽能快速迭代测试，却往往无法提供同等置信水平，也难以揭示对业务指标的影响。我们提出一种新颖、轻量且灵活的假设探究方法——场景分析，旨在利用用户数据与业务指标估算来支持产品领导者决策。其优势在于：能指导因消费增长或转移所产生的权衡取舍，预测留存率等长期结果及其他重要业务指标的变动趋势，并可大规模生成指标间关联性的假设。