Feature selection aims to identify the optimal feature subset for enhancing downstream models. Effective feature selection can remove redundant features, save computational resources, accelerate the model learning process, and improve the model overall performance. However, existing works are often time-intensive to identify the effective feature subset within high-dimensional feature spaces. Meanwhile, these methods mainly utilize a single downstream task performance as the selection criterion, leading to the selected subsets that are not only redundant but also lack generalizability. To bridge these gaps, we reformulate feature selection through a neuro-symbolic lens and introduce a novel generative framework aimed at identifying short and effective feature subsets. More specifically, we found that feature ID tokens of the selected subset can be formulated as symbols to reflect the intricate correlations among features. Thus, in this framework, we first create a data collector to automatically collect numerous feature selection samples consisting of feature ID tokens, model performance, and the measurement of feature subset redundancy. Building on the collected data, an encoder-decoder-evaluator learning paradigm is developed to preserve the intelligence of feature selection into a continuous embedding space for efficient search. Within the learned embedding space, we leverage a multi-gradient search algorithm to find more robust and generalized embeddings with the objective of improving model performance and reducing feature subset redundancy. These embeddings are then utilized to reconstruct the feature ID tokens for executing the final feature selection. Ultimately, comprehensive experiments and case studies are conducted to validate the effectiveness of the proposed framework.

翻译：特征选择旨在识别最优特征子集以增强下游模型。有效的特征选择能够去除冗余特征、节省计算资源、加速模型学习过程并提升模型整体性能。然而现有方法在高维特征空间中识别有效特征子集时往往耗时较长。同时，这些方法主要采用单一下游任务性能作为选择准则，导致所选子集不仅存在冗余性，更缺乏泛化能力。为弥补这些不足，我们通过神经符号视角重新构建特征选择问题，提出一种新型生成式框架以识别短小有效的特征子集。具体而言，我们发现所选特征子集的ID标记可被形式化为反映特征间复杂关联的符号。因此在该框架中，我们首先构建数据采集器，自动收集包含特征ID标记、模型性能及特征子集冗余度测量值的特征选择样本。基于采集数据，我们开发了编码器-解码器-评估器学习范式，将特征选择智能保留至连续嵌入空间以实现高效搜索。在习得的嵌入空间中，我们采用多梯度搜索算法，以提升模型性能与降低特征子集冗余度为双目标，寻找更具鲁棒性和泛化性的嵌入表示。最终将此类嵌入用于重构特征ID标记以执行特征选择。最后通过综合实验与案例研究验证了所提框架的有效性。