Hyperparameter tuning plays a crucial role in optimizing the performance of predictive learners. Cross--validation (CV) is a widely adopted technique for estimating the error of different hyperparameter settings. Repeated cross-validation (RCV) has been commonly employed to reduce the variability of CV errors. In this paper, we introduce a novel approach called blocked cross-validation (BCV), where the repetitions are blocked with respect to both CV partition and the random behavior of the learner. Theoretical analysis and empirical experiments demonstrate that BCV provides more precise error estimates compared to RCV, even with a significantly reduced number of runs. We present extensive examples using real--world data sets to showcase the effectiveness and efficiency of BCV in hyperparameter tuning. Our results indicate that BCV outperforms RCV in hyperparameter tuning, achieving greater precision with fewer computations.

翻译：超参数调优在优化预测学习器的性能中起着关键作用。交叉验证（CV）是一种广泛采用的技术，用于估计不同超参数设置的误差。重复交叉验证（RCV）常被用来降低CV误差的变异性。本文提出了一种名为块交叉验证（BCV）的新方法，其中重复过程在CV分区和学习器的随机行为两方面均被分块。理论分析和实证实验表明，即使在运行次数显著减少的情况下，BCV相比RCV能提供更精确的误差估计。我们通过使用真实数据集的大量示例，展示了BCV在超参数调优中的有效性和效率。结果表明，在超参数调优中，BCV以更少的计算量实现了更高的精确度，从而优于RCV。