High-dimensional, underdetermined and highly correlated systems are common in data science practice, especially when analyzing physical measurements. In such settings, feature selection poses a fundamental challenge because multiple distinct sparse subsets may explain the response equally well. Their identification is crucial not only for predictive modeling but also for generating domain-specific insights into the underlying mechanisms. Yet, conventional methods typically isolate a single solution, obscuring the full spectrum of plausible explanations. This work introduces GEMSS (Gaussian Ensemble for Multiple Sparse Solutions), a variational algorithm designed to simultaneously discover multiple, diverse sparse feature combinations. The method employs a structured spike-and-slab prior for sparsity, a mixture of Gaussians to approximate the intractable multimodal posterior, and a Jaccard-based penalty to further control solution diversity. A single objective function is optimized via stochastic gradient descent. The method is tested on 128 comprehensive experiments by a novel benchmarking framework designed to generate artificial problems with multiple sparse solutions of equal predictive properties. This allows us to measure the retrieval of ground truth features rather than only evaluating predictive performance -- characteristics more fitting to our practical needs. A comparative analysis shows that GEMSS consistently outperforms five prominent feature selection methods adapted through the ALFESE framework. Finally, we demonstrate practical usability through 3 challenging real-world datasets from metabolomics and physical chemistry: GEMSS successfully isolates multiple distinct yet quality solutions. GEMSS is available as a PyPI package 'gemss'. The corresponding repository github.com/kat-er-ina/gemss/ includes the full codebase and a free, no-code application GEMSS Explorer.

翻译：高维、欠定且高度相关的系统在数据科学实践中普遍存在，尤其在分析物理测量数据时尤为突出。在此类场景下，特征选择面临根本性挑战，因为多个不同的稀疏子集可能对响应变量具有同等程度的解释能力。识别这些子集不仅对预测建模至关重要，更能为理解潜在机制提供领域特异性见解。然而，传统方法通常仅能孤立地给出单一解，掩盖了合理解释的全谱系。本研究提出GEMSS（高斯集成多稀疏解法）——一种旨在同时发现多个多样稀疏特征组合的变分算法。该方法采用结构化尖峰-平板先验实现稀疏性，通过高斯混合模型近似难以处理的峰状后验分布，并引入基于杰卡德系数的惩罚项进一步控制解的多样性。通过随机梯度下降优化单一目标函数。我们采用新型基准测试框架在128项综合实验中验证该方法，该框架可生成具有同等预测特性的多稀疏解人工问题，从而能够直接度量真实特征的召回率而不仅评估预测性能——这更符合实际需求。对比分析表明，GEMSS始终优于通过ALFESE框架适配的五种主流特征选择方法。最后，我们通过代谢组学和物理化学领域的三个具有挑战性的真实数据集展示了其实际可用性：GEMSS成功分离出多个不同且质量可靠的解。GEMSS已作为PyPI包'gemss'发布，对应仓库github.com/kat-er-ina/gemss/包含完整代码库及免费无代码应用GEMSS Explorer。