We extend a heuristic method for automatic dimensionality selection, which maximizes a profile likelihood to identify "elbows" in scree plots. Our extension enables researchers to make automatic choices of multiple hyper-parameters simultaneously. To facilitate our extension to multi-dimensions, we propose a "softened" profile likelihood. We present two distinct parameterizations of our solution and demonstrate our approach on elastic nets, support vector machines, and neural networks. We also report a small simulation study to investigate violations to an assumption we make, and briefly discuss applications of our method to other data-analytic tasks than hyper-parameter selection.
翻译:我们扩展了一种用于自动维度选择的启发式方法,该方法通过最大化轮廓似然来识别拐点图(scree plot)中的“肘部”。我们的扩展使研究人员能够同时自动选择多个超参数。为促进这一多维扩展的实现,我们提出了一种“软化”的轮廓似然。我们展示了所提出解的两种不同参数化形式,并在弹性网络、支持向量机和神经网络上验证了我们的方法。此外,我们通过一项小型模拟研究考察了所设假设被违反的情况,并简要讨论了该方法在超参数选择之外的其他数据分析任务中的应用。