Randomized experiments (or A/B tests) are widely used to evaluate interventions in dynamic systems such as recommendation platforms, marketplaces, and digital health. In these settings, interventions affect both current and future system states, so estimating the global average treatment effect (GATE) requires accounting for temporal dynamics, which is especially challenging in the presence of nonstationarity; existing approaches suffer from high bias, high variance, or both. In this paper, we address this challenge via the novel Truncated Policy Gradient (TPG) estimator, which replaces instantaneous outcomes with short-horizon outcome trajectories. The estimator admits a policy gradient interpretation: it is a truncation of the first-order approximation to the GATE, yielding provable reductions in bias and variance in nonstationary Markovian settings. We further establish a central limit theorem for the TPG estimator and develop a consistent variance estimator that remains valid under nonstationarity with single-trajectory data. We validate our theory with two real-world case studies. The results show that relative to existing approaches, a well-calibrated TPG estimator can achieve a favorable balance between bias and variance in nonstationary settings, highlighting the value of the policy-gradient perspective for designing effective estimators under complex dynamics.

翻译：随机实验（或A/B测试）广泛用于评估推荐系统、市场平台及数字健康等动态系统中的干预措施。在此类场景中，干预措施不仅影响当前系统状态，还会波及未来状态，因此估计全局平均处理效应（GATE）需考虑时间动态特性——尤其当系统存在非平稳性时更具挑战性。现有方法存在高偏差、高方差或两者兼具的缺陷。本文提出新型截断策略梯度（TPG）估计器应对该挑战，该方法用短视界结果轨迹替代瞬时结果。该估计器具有策略梯度解释：作为GATE一阶近似的截断，可证实地降低非平稳马尔可夫场景中的偏差与方差。我们进一步建立了TPG估计量的中心极限定理，并开发了在非平稳条件下利用单轨迹数据仍保持一致的方差估计量。通过两项真实世界案例研究验证理论结果，表明相较于现有方法，校准得当的TPG估计器能在非平稳场景中实现偏差与方差的理想平衡，凸显了策略梯度视角在复杂动态系统中设计有效估计器的价值。