We propose conditional PED-ANOVA (condPED-ANOVA), a principled framework for estimating hyperparameter importance (HPI) in conditional search spaces, where the presence or domain of a hyperparameter can depend on other hyperparameters. Although the original PED-ANOVA provides a fast and efficient way to estimate HPI within the top-performing regions of the search space, it assumes a fixed, unconditional search space and therefore cannot properly handle conditional hyperparameters. To address this, we introduce a conditional HPI for top-performing regions and derive a closed-form estimator that accurately reflects conditional activation and domain changes. Experiments show that naive adaptations of existing HPI estimators yield misleading or uninterpretable importance estimates in conditional settings, whereas condPED-ANOVA consistently provides meaningful importances that reflect the underlying conditional structure.
翻译:本文提出条件PED-ANOVA(condPED-ANOVA),这是一个用于估计条件搜索空间中超参数重要性(HPI)的原则性框架。在条件搜索空间中,某个超参数的存在与否或其定义域可能依赖于其他超参数。尽管原始PED-ANOVA提供了一种快速有效的方法来估计搜索空间最优性能区域内的HPI,但其假设搜索空间是固定且无条件的,因此无法正确处理条件超参数。为解决此问题,我们针对最优性能区域引入了条件HPI概念,并推导出闭式估计量,该估计量能准确反映条件激活与定义域变化。实验表明,在条件设置下,对现有HPI估计量的简单适配会产生误导性或不可解释的重要性估计,而condPED-ANOVA始终能提供反映底层条件结构的意义明确的重要性度量。