Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curvature, or combinations thereof, to better model the latent space and enhance model performance. However, little attention has been given to the problem of automatically identifying the optimal latent geometry for the downstream task. We mathematically define this novel formulation and coin it as neural latent geometry search (NLGS). More specifically, we introduce a principled method that searches for a latent geometry composed of a product of constant curvature model spaces with minimal query evaluations. To accomplish this, we propose a novel notion of distance between candidate latent geometries based on the Gromov-Hausdorff distance from metric geometry. In order to compute the Gromov-Hausdorff distance, we introduce a mapping function that enables the comparison of different manifolds by embedding them in a common high-dimensional ambient space. Finally, we design a graph search space based on the calculated distances between candidate manifolds and use Bayesian optimization to search for the optimal latent geometry in a query-efficient manner. This is a general method which can be applied to search for the optimal latent geometry for a variety of models and downstream tasks. Extensive experiments on synthetic and real-world datasets confirm the efficacy of our method in identifying the optimal latent geometry for multiple machine learning problems.

翻译：近期研究表明，通过使隐空间几何与底层数据结构对齐，可提升机器学习模型的性能。研究者不再局限于欧几里得空间，而是提出使用恒定曲率的双曲空间和球面空间或其组合，以更准确地建模隐空间并提升模型性能。然而，针对自动识别下游任务最优隐空间几何的问题，目前鲜有研究关注。我们对此新范式进行了数学定义，并将其命名为神经隐式几何搜索（NLGS）。具体而言，我们提出一种基于原则的方法，搜索由恒定曲率模型空间乘积构成的隐空间几何，且仅需最少查询次数。为实现该目标，我们基于度量几何中的格罗莫夫-豪斯多夫距离，提出了候选隐空间几何间的距离新概念。为计算格罗莫夫-豪斯多夫距离，我们引入映射函数，通过将不同流形嵌入公共高维空间实现其比较。最终，我们基于候选流形间的计算距离设计图搜索空间，并采用贝叶斯优化以查询高效的方式搜索最优隐空间几何。该通用方法可应用于多种模型与下游任务的最优隐空间几何搜索。在合成数据集与真实数据集上的大量实验证实，本方法能有效识别多个机器学习问题的最优隐空间几何。