Due to the challenges in acquiring paired Text-3D data and the inherent irregularity of 3D data structures, combined representation learning of 3D point clouds and text remains unexplored. In this paper, we propose a novel Riemann-based Multi-scale Attention Reasoning Network (RMARN) for text-3D retrieval. Specifically, the extracted text and point cloud features are refined by their respective Adaptive Feature Refiner (AFR). Furthermore, we introduce the innovative Riemann Local Similarity (RLS) module and the Global Pooling Similarity (GPS) module. However, as 3D point cloud data and text data often possess complex geometric structures in high-dimensional space, the proposed RLS employs a novel Riemann Attention Mechanism to reflect the intrinsic geometric relationships of the data. Without explicitly defining the manifold, RMARN learns the manifold parameters to better represent the distances between text-point cloud samples. To address the challenges of lacking paired text-3D data, we have created the large-scale Text-3D Retrieval dataset T3DR-HIT, which comprises over 3,380 pairs of text and point cloud data. T3DR-HIT contains coarse-grained indoor 3D scenes and fine-grained Chinese artifact scenes, consisting of 1,380 and over 2,000 text-3D pairs, respectively. Experiments on our custom datasets demonstrate the superior performance of the proposed method. Our code and proposed datasets are available at \url{https://github.com/liwrui/RMARN}.

翻译：由于获取成对的文本-3D数据存在挑战，以及3D数据结构固有的不规则性，3D点云与文本的联合表示学习仍未被充分探索。本文提出了一种新颖的基于黎曼的多尺度注意力推理网络（RMARN）用于文本-3D检索。具体而言，提取的文本和点云特征通过各自的自适应特征精炼器（AFR）进行优化。此外，我们引入了创新的黎曼局部相似性（RLS）模块和全局池化相似性（GPS）模块。然而，由于3D点云数据和文本数据在高维空间中通常具有复杂的几何结构，所提出的RLS采用了一种新颖的黎曼注意力机制来反映数据的内在几何关系。RMARN无需显式定义流形，而是学习流形参数以更好地表示文本-点云样本之间的距离。针对缺乏成对文本-3D数据的挑战，我们创建了大规模文本-3D检索数据集T3DR-HIT，该数据集包含超过3,380对文本和点云数据。T3DR-HIT包含粗粒度的室内3D场景和细粒度的中国文物场景，分别由1,380对和超过2,000对文本-3D数据组成。在我们自定义数据集上的实验证明了所提方法的优越性能。我们的代码和所提数据集可在 \url{https://github.com/liwrui/RMARN} 获取。