The wide adoption and significant computing resource consumption of attention-based Transformers, e.g., Vision Transformer and large language models, have driven the demands for efficient hardware accelerators. While electronic accelerators have been commonly used, there is a growing interest in exploring photonics as an alternative technology due to its high energy efficiency and ultra-fast processing speed. Optical neural networks (ONNs) have demonstrated promising results for convolutional neural network (CNN) workloads that only require weight-static linear operations. However, they fail to efficiently support Transformer architectures with attention operations due to the lack of ability to process dynamic full-range tensor multiplication. In this work, we propose a customized high-performance and energy-efficient photonic Transformer accelerator, DOTA. To overcome the fundamental limitation of existing ONNs, we introduce a novel photonic tensor core, consisting of a crossbar array of interference-based optical vector dot-product engines, that supports highly-parallel, dynamic, and full-range matrix-matrix multiplication. Our comprehensive evaluation demonstrates that DOTA achieves a >4x energy and a >10x latency reduction compared to prior photonic accelerators, and delivers over 20x energy reduction and 2 to 3 orders of magnitude lower latency compared to the electronic Transformer accelerator. Our work highlights the immense potential of photonic computing for efficient hardware accelerators, particularly for advanced machine learning workloads.

翻译：基于注意力机制的Transformer模型（如Vision Transformer和大型语言模型）的广泛应用及其对计算资源的巨大消耗，推动了高效硬件加速器的需求。尽管电子加速器已被普遍使用，但因其高能效和超快处理速度，光子技术作为替代方案正受到越来越多的关注。光学神经网络已在仅需权重静态线性操作的卷积神经网络任务中展现出有前景的结果。然而，由于缺乏处理动态全范围张量乘法的能力，它们无法高效支持包含注意力运算的Transformer架构。在本文中，我们提出了一种定制的高性能、高能效光子Transformer加速器——DOTA。为克服现有光学神经网络的根本限制，我们引入了一种新型光子张量核心，该核心由基于干涉的光学向量点积引擎交叉阵列构成，能够支持高度并行、动态且全范围的矩阵-矩阵乘法。全面的评估结果表明，与先前的光子加速器相比，DOTA实现了超过4倍的能耗降低和超过10倍的延迟缩减；与电子Transformer加速器相比，能耗降低超过20倍，延迟降低2至3个数量级。本工作突显了光子计算在高效硬件加速器（尤其是先进机器学习任务）中的巨大潜力。