Suppose a planner has a pre-trained simulator of a sequential decision problem and the option to run real experiments in the field. The simulator is cheap to query but inherits confounding and drift from its calibration data. Experimentation is unbiased but consumes one real unit per trial. We study when, and how, the planner should supplement the simulator with experiments. We give three results. First, an extended simulation lemma decomposes the simulator's value error into a calibration--deployment shift that randomization can identify and a parametric residual that no further interaction can reduce. Second, the value gap between the simulator-optimal policy and the optimum splits into a local component, on states the deployed policy already visits, and a reachability component, on states it does not. The reachability component stays bounded away from zero at any horizon under purely passive learning. Third, we propose Fisher-SEP, a simulation-aided experimental policy (SEP) that minimizes the posterior predictive variance of a target policy's value, with reward-only and transition-only specializations. Two case studies illustrate the regimes. In a vending-machine supply chain, front-loaded experimentation overtakes posterior updating once the horizon is long enough to amortize the pilot. In an HIV mobile-testing example with a corridor that separates a well-surveilled region from a poorly-surveilled one, only designed exploration reaches the poorly-surveilled region.

翻译：摘要：假设规划者拥有一个序列决策问题的预训练模拟器，并可以选择在现场进行真实实验。模拟器查询成本低廉，但其校准数据会引入混杂因素和漂移。实验虽无偏，但每次试验消耗一个真实单元。我们研究规划者何时以及如何用实验补充模拟器。我们给出三项结果：第一，扩展的仿真引理将模拟器的价值误差分解为两部分——随机化可识别的校准-部署偏移，以及任何进一步交互都无法减少的参数残差。第二，模拟器最优策略与真实最优策略之间的价值差距可分为局部成分（部署策略已访问的状态）和可达性成分（尚未访问的状态）。在纯被动学习下，可达性成分在任何时间范围内都保持非零下界。第三，我们提出 Fisher-SEP，一种模拟辅助实验策略，通过奖励专属和转移专属的变体，最小化目标策略价值的后验预测方差。两个案例研究说明了这些机制：在自动售货机供应链中，当期初试点足够长以摊销成本时，前期实验将超越后验更新；在HIV移动检测示例中，当存在分隔监控良好区域与监控薄弱区域的走廊时，只有经过设计的探索才能到达监控薄弱区域。