Evaluation metrics for prediction error, model selection and model averaging on space-time data are understudied and poorly understood. The absence of independent replication makes prediction ambiguous as a concept and renders evaluation procedures developed for independent data inappropriate for most space-time prediction problems. Motivated by air pollution data collected during California wildfires in 2008, this manuscript attempts a formalization of the true prediction error associated with spatial interpolation. We investigate a variety of cross-validation (CV) procedures employing both simulations and case studies to provide insight into the nature of the estimand targeted by alternative data partition strategies. Consistent with recent best practice, we find that location-based cross-validation is appropriate for estimating spatial interpolation error as in our analysis of the California wildfire data. Interestingly, commonly held notions of bias-variance trade-off of CV fold size do not trivially apply to dependent data, and we recommend leave-one-location-out (LOLO) CV as the preferred prediction error metric for spatial interpolation.

翻译：由于对预测错误、模型选择和平均空间时间数据模型的评价指标没有进行充分研究,也没有很好地理解。由于缺乏独立复制,预测作为一个概念变得模糊不清,使为独立数据制定的评价程序不适合大多数空间时间预测问题。2008年加利福尼亚野火期间收集的空气污染数据促使这一手稿试图正式确定与空间内插有关的真实预测错误。我们调查了各种交叉校准(CV)程序,采用模拟和案例研究,以深入了解替代数据分割战略所针对天平和对象的性质。根据最近的最佳做法,我们认为基于地点的交叉校准适合于估计空间内插错误,正如我们在分析加利福尼亚野火数据时所发现的那样。有趣的是,常见的CV折叠大小偏差权衡概念并不会轻描淡地适用于依赖的数据,我们建议将允许一地放置(LOLO)CV(LO)CV) CV(CV)作为空间内插图的首选预测错误指标。