Multiplication is a fundamental operation in many applications, and multipliers are widely adopted in various circuits. However, optimizing multipliers is challenging and non-trivial due to the huge design space. In this paper, we propose RL-MUL, a multiplier design optimization framework based on reinforcement learning. Specifically, we utilize matrix and tensor representations for the compressor tree of a multiplier, based on which the convolutional neural networks can be seamlessly incorporated as the agent network. The agent can learn to optimize the multiplier structure based on a Pareto-driven reward which is customized to accommodate the trade-off between area and delay. Additionally, the capability of RL-MUL is extended to optimize the fused multiply-accumulator (MAC) designs. Experiments are conducted on different bit widths of multipliers. The results demonstrate that the multipliers produced by RL-MUL can dominate all baseline designs in terms of area and delay. The performance gain of RL-MUL is further validated by comparing the area and delay of processing element arrays using multipliers from RL-MUL and baseline approaches.

翻译：乘法是众多应用中的基本运算，乘法器被广泛用于各类电路中。然而，由于设计空间巨大，乘法器优化具有挑战性且不容小觑。本文提出RL-MUL，一种基于强化学习的乘法器设计优化框架。具体而言，我们利用矩阵和张量表示乘法器的压缩树，基于此，卷积神经网络可无缝集成为智能体网络。该智能体能够基于帕累托驱动的奖励学习优化乘法器结构，该奖励根据面积和延迟之间的权衡进行定制。此外，RL-MUL的能力被扩展以优化融合乘累加器（MAC）设计。实验在不同位宽的乘法器上进行。结果表明，RL-MUL生成的乘法器在面积和延迟方面能够主导所有基线设计。通过比较使用RL-MUL和基线方法生成乘法器的处理单元阵列的面积和延迟，进一步验证了RL-MUL的性能优势。