Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curvature, or combinations thereof, to better model the latent space and enhance model performance. However, little attention has been given to the problem of automatically identifying the optimal latent geometry for the downstream task. We mathematically define this novel formulation and coin it as neural latent geometry search (NLGS). More specifically, we introduce an initial attempt to search for a latent geometry composed of a product of constant curvature model spaces with a small number of query evaluations, under some simplifying assumptions. To accomplish this, we propose a novel notion of distance between candidate latent geometries based on the Gromov-Hausdorff distance from metric geometry. In order to compute the Gromov-Hausdorff distance, we introduce a mapping function that enables the comparison of different manifolds by embedding them in a common high-dimensional ambient space. We then design a graph search space based on the notion of smoothness between latent geometries and employ the calculated distances as an additional inductive bias. Finally, we use Bayesian optimization to search for the optimal latent geometry in a query-efficient manner. This is a general method which can be applied to search for the optimal latent geometry for a variety of models and downstream tasks. We perform experiments on synthetic and real-world datasets to identify the optimal latent geometry for multiple machine learning problems.

翻译：近期研究表明，通过将隐空间几何与底层数据结构对齐可提升机器学习模型的性能。研究者不再仅依赖欧几里得空间，而是提出使用具有恒定曲率的双曲空间和球面空间或其组合来更好地建模隐空间，从而增强模型性能。然而，针对下游任务自动识别最优隐几何的问题鲜有关注。我们从数学上定义了这一新范式，并将其命名为神经隐式几何搜索（NLGS）。具体而言，我们提出一种初步尝试：在若干简化假设下，以少量查询评估次数搜索由恒定曲率模型空间的乘积构成的隐几何。为此，我们基于度量几何中的格罗莫夫-豪斯多夫距离，提出一种候选隐几何间距离的新概念。为计算该距离，我们引入映射函数，通过将不同流形嵌入公共高维空间实现其比较。随后，基于隐几何间的平滑性概念设计图搜索空间，并将计算所得距离作为额外归纳偏置。最终采用贝叶斯优化以高效查询方式搜索最优隐几何。该方法具有通用性，可应用于多种模型及下游任务的最优隐几何搜索。我们在合成数据集和真实数据集上开展了实验，为多个机器学习问题识别最优隐几何。