Reliable estimation of predictive performance is essential for spatial environmental modeling, where machine-learning models are used to generate maps from unevenly distributed observations. Standard cross-validation (CV) assumes that validation data are representative of prediction conditions across the target domain. In practice, this assumption is often violated due to preferential or clustered sampling, leading to biased performance and uncertainty estimates. We introduce a deployment-oriented validation framework based on weighted CV that aligns validation tasks with the distribution of prediction tasks across a specified domain. The framework includes importance-weighted cross-validation (IWCV) and a calibration-based approach, Target-Weighted Cross-Validation (TWCV), which uses spatially meaningful task descriptors such as environmental covariates and prediction distance. Simulation experiments show that conventional non-spatial and spatial CV strategies can exhibit substantial bias under realistic sampling designs, whereas weighted CV approaches substantially reduce this bias when validation tasks adequately cover the deployment-task space. A case study on mapping nitrogen dioxide (NO$_2$) concentrations across Germany demonstrates that standard CV can overestimate prediction error due to sampling bias, while weighted CV yields estimates more consistent with deployment conditions. The framework separates validation task generation from risk estimation and provides a practical approach for improving performance assessment in spatial prediction settings where sample distributions differ from prediction domains.
翻译:可靠的预测性能估计对空间环境建模至关重要,此类模型利用机器学习技术基于非均匀分布的观测生成地图。标准交叉验证假设验证数据能够代表目标域中的预测条件,然而在实际应用中,由于偏好性采样或聚类采样,该假设常被违反,导致性能估计与不确定性估计出现偏差。本文提出一种基于加权交叉验证的面向部署验证框架,通过将验证任务与指定域内的预测任务分布对齐,实现性能评估的优化。该框架包含重要性加权交叉验证及基于校准的目标加权交叉验证方法,后者采用环境协变量、预测距离等具有空间意义的任务描述符。模拟实验表明,在现实采样设计下,传统非空间与空间交叉验证策略可能产生显著偏差,而加权交叉验证方法在验证任务充分覆盖部署任务空间时可大幅降低该偏差。基于德国二氧化氮浓度填图案例研究发现,标准交叉验证因采样偏差可能高估预测误差,而加权交叉验证产生的估计更符合部署条件。该框架将验证任务生成与风险估计解耦,为样本分布与预测域不一致的空间预测场景提供了一种改进性能评估的实用方法。