In this paper, we propose a method for calculating the three-dimensional (3D) position and orientation of a pallet placed on a shelf on the side of a forklift truck using a 360-degree camera. By using a 360-degree camera mounted on the forklift truck, it is possible to observe both the pallet at the side of the forklift and one several meters ahead. However, the pallet on the obtained image is observed with different distortion depending on its 3D position, so that it is difficult to extract the pallet from the image. To solve this problem, a method [1] has been proposed for detecting a pallet by projecting a 360-degree image on a vertical plane that coincides with the front of the shelf to calculate an image similar to the image seen from the front of the shelf. At the same time as the detection, the approximate position and orientation of the detected pallet can be obtained, but the accuracy is not sufficient for automatic control of the forklift truck. In this paper, we propose a method for accurately detecting the yaw angle, which is the angle of the front surface of the pallet in the horizontal plane, by projecting the 360-degree image on a horizontal plane including the boundary line of the front surface of the detected pallet. The position of the pallet is also determined by moving the vertical plane having the detected yaw angle back and forth, and finding the position at which the degree of coincidence between the projection image on the vertical plane and the actual size of the front surface of the pallet is maximized. Experiments using real images taken in a laboratory and an actual warehouse have confirmed that the proposed method can calculate the position and orientation of a pallet within a reasonable calculation time and with the accuracy necessary for inserting the fork into the hole in the front of the pallet.

翻译：本文提出一种利用360度相机计算叉车侧面货架上托盘三维位姿的方法。通过在叉车上安装360度相机，可同时观测叉车侧方托盘及前方数米处托盘。然而，由于获取图像中托盘的畸变程度随其三维位置变化，导致难以从图像中直接提取托盘。为解决该问题，已有方法[1]通过将360度图像投影至与货架正面重合的垂直平面，计算类似货架正面视角的图像，并在检测托盘的同时获取其近似位姿，但精度尚不足以支持叉车自动控制。本文提出通过将360度图像投影至包含检测托盘前表面边界的水平面，精确检测表征托盘前表面水平角度的偏航角。进而通过前后移动具有该偏航角的垂直平面，寻找垂直平面投影图像与托盘前表面实际尺寸吻合度最高的位置，确定托盘空间位置。采用实验室及真实仓库场景的实拍图像进行实验，结果表明：所提方法可在合理计算时间内，以叉齿插入托盘前表面孔洞所需的精度，计算出托盘位姿。