Estimating geometric elements such as depth, camera motion, and optical flow from images is an important part of the robot's visual perception. We use a joint self-supervised method to estimate the three geometric elements. Depth network, optical flow network and camera motion network are independent of each other but are jointly optimized during training phase. Compared with independent training, joint training can make full use of the geometric relationship between geometric elements and provide dynamic and static information of the scene. In this paper, we improve the joint self-supervision method from three aspects: network structure, dynamic object segmentation, and geometric constraints. In terms of network structure, we apply the attention mechanism to the camera motion network, which helps to take advantage of the similarity of camera movement between frames. And according to attention mechanism in Transformer, we propose a plug-and-play convolutional attention module. In terms of dynamic object, according to the different influences of dynamic objects in the optical flow self-supervised framework and the depth-pose self-supervised framework, we propose a threshold algorithm to detect dynamic regions, and mask that in the loss function respectively. In terms of geometric constraints, we use traditional methods to estimate the fundamental matrix from the corresponding points to constrain the camera motion network. We demonstrate the effectiveness of our method on the KITTI dataset. Compared with other joint self-supervised methods, our method achieves state-of-the-art performance in the estimation of pose and optical flow, and the depth estimation has also achieved competitive results. Code will be available https://github.com/jianfenglihg/Unsupervised_geometry.

翻译：测算深度、 摄像运动和图像光学流等几何要素是机器人视觉感知的一个重要部分。 我们使用联合自我监督的方法来估计三个几何元素。 深度网络、 光学流网络和摄影机运动网络彼此独立, 在培训阶段共同优化。 与独立培训相比, 联合培训可以充分利用几何元素之间的几何关系, 并提供动态和静态的场景信息。 在本文中, 我们从三个方面改进了联合自我监督方法: 网络结构、 动态物体分割和几何限制。 在网络结构方面, 我们将关注机制应用到相机运动网络, 这有助于利用各框架之间相近的摄像运动。 根据变异器的注意机制, 我们建议一个插插和播放动的演动关注模块。 在动态物体方面, 根据光学流自我监督框架和深度自我监督框架的不同影响, 我们提出一个门槛值算法, 检测动态区域, 并用传统的光学数据流流流数据流流, 也用常规的自我评估方法 。