Merging satellite and gauge data with machine learning produces high-resolution precipitation datasets, but uncertainty estimates are often missing. We addressed the gap of how to optimally provide such estimates by benchmarking six algorithms, mostly novel even for the more general task of quantifying predictive uncertainty in spatial prediction settings. On 15 years of monthly data from over the contiguous United States (CONUS), we compared quantile regression (QR), quantile regression forests (QRF), generalized random forests (GRF), gradient boosting machines (GBM), light gradient boosting machine (LightGBM), and quantile regression neural networks (QRNN). Their ability to issue predictive precipitation quantiles at nine quantile levels (0.025, 0.050, 0.100, 0.250, 0.500, 0.750, 0.900, 0.950, 0.975), approximating the full probability distribution, was evaluated using quantile scoring functions and the quantile scoring rule. Predictors at a site were nearby values from two satellite precipitation retrievals, namely PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) and IMERG (Integrated Multi-satellitE Retrievals), and the site's elevation. The dependent variable was the monthly mean gauge precipitation. With respect to QR, LightGBM showed improved performance in terms of the quantile scoring rule by 11.10%, also surpassing QRF (7.96%), GRF (7.44%), GBM (4.64%) and QRNN (1.73%). Notably, LightGBM outperformed all random forest variants, the current standard in spatial prediction with machine learning. To conclude, we propose a suite of machine learning algorithms for estimating uncertainty in spatial data prediction, supported with a formal evaluation framework based on scoring functions and scoring rules.
翻译:通过机器学习融合卫星与站点数据可生成高分辨率降水数据集,但通常缺乏不确定性估计。本研究通过基准测试六种算法填补了如何优化提供此类估计的空白,其中多数算法即使在更广义的空间预测场景中量化预测不确定性任务上也具有新颖性。基于美国本土连续15年月尺度数据,我们比较了分位数回归(QR)、分位数回归森林(QRF)、广义随机森林(GRF)、梯度提升机(GBM)、轻量梯度提升机(LightGBM)和分位数回归神经网络(QRNN)。这些算法在九个分位数水平(0.025、0.050、0.100、0.250、0.500、0.750、0.900、0.950、0.975)上发布预测降水分位数的能力——近似完整概率分布——通过分位数评分函数和分位数评分规则进行评估。站点预测因子包括来自两种卫星降水反演产品(即PERSIANN和IMERG)的邻近观测值以及站点高程。因变量为月均站点降水量。相较于QR,LightGBM在分位数评分规则方面表现出11.10%的性能提升,同时超越QRF(7.96%)、GRF(7.44%)、GBM(4.64%)和QRNN(1.73%)。值得注意的是,LightGBM在所有随机森林变体(当前机器学习空间预测的标准方法)中均表现更优。最后,我们提出了一套用于估计空间数据预测不确定性的机器学习算法,并建立了基于评分函数与评分规则的正式评估框架作为支撑。