The structured light (SL)-based 3D measurement techniques with deep learning have been widely studied, among which speckle projection profilometry (SPP) and fringe projection profilometry (FPP) are two popular methods. However, they generally use a single projection pattern for reconstruction, resulting in fringe order ambiguity or poor reconstruction accuracy. To alleviate these problems, we propose a parallel dual-branch Convolutional Neural Network (CNN)-Transformer network (PDCNet), to take advantage of convolutional operations and self-attention mechanisms for processing different SL modalities. Within PDCNet, a Transformer branch is used to capture global perception in the fringe images, while a CNN branch is designed to collect local details in the speckle images. To fully integrate complementary features, we design a double-stream attention aggregation module (DAAM) that consist of a parallel attention subnetwork for aggregating multi-scale spatial structure information. This module can dynamically retain local and global representations to the maximum extent. Moreover, an adaptive mixture density head with bimodal Gaussian distribution is proposed for learning a representation that is precise near discontinuities. Compared to the standard disparity regression strategy, this adaptive mixture head can effectively improves performance at object boundaries. Extensive experiments demonstrate that our method can reduce fringe order ambiguity while producing high-accuracy results on a self-made dataset. We also show that the proposed architecture reveals the potential in infrared-visible image fusion task.

翻译：基于结构光的三维测量技术与深度学习相结合已得到广泛研究，其中散斑投影轮廓术和条纹投影轮廓术是两种主流方法。然而，这些方法通常采用单一投影图案进行重建，导致条纹级次模糊或重建精度不足。为缓解这些问题，我们提出一种并行双分支卷积神经网络-Transformer网络，以融合卷积运算与自注意力机制的优势来处理不同结构光模态。在该网络中，Transformer分支用于捕捉条纹图像的全局感知特征，而CNN分支则专门提取散斑图像的局部细节特征。为实现互补特征的充分融合，我们设计了双流注意力聚合模块，该模块包含并行注意力子网络以聚合多尺度空间结构信息，能够动态最大化保留局部与全局表征。此外，我们提出采用双峰高斯分布的自适应混合密度头，用于学习在边缘不连续区域保持精确的表征。相较于标准视差回归策略，该自适应混合头能有效提升物体边界处的性能表现。大量实验表明，我们的方法在自制数据集上能够显著降低条纹级次模糊度，同时实现高精度重建结果。我们还验证了所提架构在红外-可见光图像融合任务中具有应用潜力。