The Bin Packing Problem (BPP) has attracted enthusiastic research interest recently, owing to widespread applications in logistics and warehousing environments. It is truly essential to optimize the bin packing to enable more objects to be packed into boxes. Object packing order and placement strategy are the two crucial optimization objectives of the BPP. However, existing optimization methods for BPP, such as the genetic algorithm (GA), emerge as the main issues in highly computational cost and relatively low accuracy, making it difficult to implement in realistic scenarios. To well relieve the research gaps, we present a novel optimization methodology of two-dimensional (2D)-BPP and three-dimensional (3D)-BPP for objects with regular shapes via deep reinforcement learning (DRL), maximizing the space utilization and minimizing the usage number of boxes. First, an end-to-end DRL neural network constructed by a modified Pointer Network consisting of an encoder, a decoder and an attention module is proposed to achieve the optimal object packing order. Second, conforming to the top-down operation mode, the placement strategy based on a height map is used to arrange the ordered objects in the boxes, preventing the objects from colliding with boxes and other objects in boxes. Third, the reward and loss functions are defined as the indicators of the compactness, pyramid, and usage number of boxes to conduct the training of the DRL neural network based on an on-policy actor-critic framework. Finally, a series of experiments are implemented to compare our method with conventional packing methods, from which we conclude that our method outperforms these packing methods in both packing accuracy and efficiency.

翻译：箱体包装问题（BPP）凭借其在物流和仓储环境中的广泛应用而引发了近期研究的热潮，优化箱体包装以实现更多物品装入箱内至关重要。物品包装顺序与放置策略是BPP的两大核心优化目标。然而，现有BPP优化方法（如遗传算法（GA））存在计算成本高、准确率相对较低等主要问题，难以在实际场景中落地。为有效填补研究空白，我们提出了一种针对规则形状物品的二维（2D）-BPP和三维（3D）-BPP新型优化方法，该方法基于深度强化学习（DRL），旨在最大化空间利用率并最小化箱体使用数量。首先，构建了一种由改进型指针网络（含编码器、解码器和注意力模块）构成的端到端DRL神经网络，用于获取最优物品包装顺序。其次，遵循自上而下的操作模式，采用基于高度图的放置策略将有序物品布置于箱体内，防止物品与箱体及箱内其他物品发生碰撞。再次，将奖励函数和损失函数定义为紧凑度、金字塔形态及箱体使用数量的指标，基于同策略的演员-评论家框架对DRL神经网络进行训练。最后，通过系列实验将我们的方法与常规包装方法进行对比，结果表明本方法在包装准确率和效率上均优于这些现有方法。