Randomized algorithms exploit stochasticity to reduce computational complexity. One important example is random feature regression (RFR) that accelerates Gaussian process regression (GPR). RFR approximates an unknown function with a random neural network whose hidden weights and biases are sampled from a probability distribution. Only the final output layer is fit to data. In randomized algorithms like RFR, the hyperparameters that characterize the sampling distribution greatly impact performance, yet are not directly accessible from samples. This makes optimization of hyperparameters via standard (gradient-based) optimization tools inapplicable. Inspired by Bayesian ideas from GPR, this paper introduces a random objective function that is tailored for hyperparameter tuning of vector-valued random features. The objective is minimized with ensemble Kalman inversion (EKI). EKI is a gradient-free particle-based optimizer that is scalable to high-dimensions and robust to randomness in objective functions. A numerical study showcases the new black-box methodology to learn hyperparameter distributions in several problems that are sensitive to the hyperparameter selection: two global sensitivity analyses, integrating a chaotic dynamical system, and solving a Bayesian inverse problem from atmospheric dynamics. The success of the proposed EKI-based algorithm for RFR suggests its potential for automated optimization of hyperparameters arising in other randomized algorithms.

翻译：随机算法利用随机性降低计算复杂度。随机特征回归（RFR）作为重要范例，可加速高斯过程回归（GPR）。RFR通过随机神经网络逼近未知函数，其隐藏层权重与偏置从概率分布中采样，仅最终输出层通过数据拟合。在RFR这类随机算法中，表征采样分布的超参数对性能影响显著，却无法直接从样本中获取，这使得基于标准（梯度）优化工具的超参数优化方法难以适用。受GPR贝叶斯思想启发，本文提出专为向量值随机特征超参数调优设计的随机目标函数，并采用集成卡尔曼反演（EKI）进行最小化。EKI作为一种无梯度、基于粒子的优化器，具备高维可扩展性且对目标函数随机性具有鲁棒性。数值研究通过四类对超参数选择敏感的问题展示了该黑箱方法学习超参数分布的能力：两项全局敏感性分析、混沌动力系统积分以及大气动力学贝叶斯反问题求解。基于EKI的RFR算法成功实施，表明该方法在其他随机算法超参数自动化优化中具有应用潜力。