In many recent works, the potential of Exploratory Landscape Analysis (ELA) features to numerically characterize, in particular, single-objective continuous optimization problems has been demonstrated. These numerical features provide the input for all kinds of machine learning tasks on continuous optimization problems, ranging, i.a., from High-level Property Prediction to Automated Algorithm Selection and Automated Algorithm Configuration. Without ELA features, analyzing and understanding the characteristics of single-objective continuous optimization problems is -- to the best of our knowledge -- very limited. Yet, despite their usefulness, as demonstrated in several past works, ELA features suffer from several drawbacks. These include, in particular, (1.) a strong correlation between multiple features, as well as (2.) its very limited applicability to multi-objective continuous optimization problems. As a remedy, recent works proposed deep learning-based approaches as alternatives to ELA. In these works, e.g., point-cloud transformers were used to characterize an optimization problem's fitness landscape. However, these approaches require a large amount of labeled training data. Within this work, we propose a hybrid approach, Deep-ELA, which combines (the benefits of) deep learning and ELA features. Specifically, we pre-trained four transformers on millions of randomly generated optimization problems to learn deep representations of the landscapes of continuous single- and multi-objective optimization problems. Our proposed framework can either be used out-of-the-box for analyzing single- and multi-objective continuous optimization problems, or subsequently fine-tuned to various tasks focussing on algorithm behavior and problem understanding.

翻译：近年来多项研究表明，探索性景观分析（ELA）特征在数值化表征单目标连续优化问题方面展现出巨大潜力。这些数值特征为连续优化问题的各类机器学习任务提供输入，涵盖高层属性预测、自动化算法选择及自动化算法配置等多个领域。据我们所知，若不借助ELA特征，对单目标连续优化问题特性的分析与理解将受到极大限制。然而，尽管过往研究已证实其有效性，ELA特征仍存在若干缺陷，主要包括：（1）多个特征间存在强相关性；（2）对多目标连续优化问题的适用性极为有限。作为改进方案，近期研究提出了基于深度学习的方法作为ELA的替代方案，例如采用点云Transformer来表征优化问题的适应度景观。但此类方法需要大量带标签的训练数据。本研究提出一种混合方法Deep-ELA，将深度学习与ELA特征的优势相结合。具体而言，我们在数百万个随机生成的优化问题上预训练了四个Transformer模型，以学习连续单目标及多目标优化问题景观的深度表征。所提出的框架既可直接用于分析单目标与多目标连续优化问题，也可针对算法行为分析与问题理解等不同任务进行微调。