In statistics, it is common to encounter multi-modal and non-smooth likelihood (or objective function) maximization problems, where the parameters have known upper and lower bounds. This paper proposes a novel derivative-free global optimization technique that can be used to solve those problems even when the objective function is not known explicitly or its derivatives are difficult or expensive to obtain. The technique is based on the pattern search algorithm, which has been shown to be effective for black-box optimization problems. The proposed algorithm works by iteratively generating new solutions from the current solution. The new solutions are generated by making movements along the coordinate axes of the constrained sample space. Before making a jump from the current solution to a new solution, the objective function is evaluated at several neighborhood points around the current solution. The best solution point is then chosen based on the objective function values at those points. Parallel threading can be used to make the algorithm more scalable. The performance of the proposed method is evaluated by optimizing up to 5000-dimensional multi-modal benchmark functions. The proposed algorithm is shown to be up to 40 and 368 times faster than genetic algorithm (GA) and simulated annealing (SA), respectively. The proposed method is also used to estimate the optimal biomarker combination from Alzheimer's disease data by maximizing the empirical estimates of the area under the receiver operating characteristic curve (AUC), outperforming the contextual popular alternative, known as step-down algorithm.

翻译：在统计学中，常会遇到多模态且非光滑的似然（或目标函数）最大化问题，其中参数具有已知的上下界。本文提出了一种新型无导数全局优化技术，即使在目标函数无法显式表达或其导数难以获取或计算代价高昂的情况下，也能解决此类问题。该技术基于模式搜索算法，该算法已被证明对黑箱优化问题有效。所提算法通过从当前解迭代生成新解来运作：新解通过沿约束样本空间的坐标轴移动生成。在从当前解跳转至新解前，需评估当前解周围若干邻域点的目标函数值，并基于这些点的函数值选择最优解。并行线程技术可用于提升算法的可扩展性。通过优化最高5000维多模态基准函数评估了所提方法的性能，结果显示该算法比遗传算法和模拟退火分别快达40倍和368倍。此外，该方法还被用于通过最大化受试者工作特征曲线下面积的经验估计值，从阿尔茨海默病数据中估计最佳生物标志物组合，其性能优于流行的上下文替代方法——逐步下降算法。