Hyperparameters play a critical role in machine learning. Hyperparameter tuning can make the difference between state-of-the-art and poor prediction performance for any algorithm, but it is particularly challenging for structure learning due to its unsupervised nature. As a result, hyperparameter tuning is often neglected in favour of using the default values provided by a particular implementation of an algorithm. While there have been numerous studies on performance evaluation of causal discovery algorithms, how hyperparameters affect individual algorithms, as well as the choice of the best algorithm for a specific problem, has not been studied in depth before. This work addresses this gap by investigating the influence of hyperparameters on causal structure learning tasks. Specifically, we perform an empirical evaluation of hyperparameter selection for some seminal learning algorithms on datasets of varying levels of complexity. We find that, while the choice of algorithm remains crucial to obtaining state-of-the-art performance, hyperparameter selection in ensemble settings strongly influences the choice of algorithm, in that a poor choice of hyperparameters can lead to analysts using algorithms which do not give state-of-the-art performance for their data.

翻译：超参数在机器学习中扮演着关键角色。超参数调优可能决定任何算法能否达到最佳或较差的预测性能，但由于结构学习的无监督特性，这一过程尤为困难。因此，超参数调优常被忽视，取而代之的是使用特定算法实现提供的默认参数值。尽管已有大量研究评估因果发现算法的性能，但超参数如何影响单个算法以及如何为特定问题选择最佳算法，此前尚未得到深入探讨。本研究通过探究超参数对因果结构学习任务的影响来填补这一空白。具体而言，我们在不同复杂度的数据集上，对一些经典学习算法的超参数选择进行了实证评估。研究发现，尽管算法选择对于获得最佳性能仍至关重要，但集成设置中的超参数选择会强烈影响算法选择，即超参数选择不当可能导致分析者使用了在自身数据上无法达到最佳性能的算法。