Targeting vision applications at the edge, in this work, we systematically explore and propose a high-performance and energy-efficient Optical In-Sensor Accelerator architecture called OISA for the first time. Taking advantage of the promising efficiency of photonic devices, the OISA intrinsically implements a coarse-grained convolution operation on the input frames in an innovative minimum-conversion fashion in low-bit-width neural networks. Such a design remarkably reduces the power consumption of data conversion, transmission, and processing in the conventional cloud-centric architecture as well as recently-presented edge accelerators. Our device-to-architecture simulation results on various image data-sets demonstrate acceptable accuracy while OISA achieves 6.68 TOp/s/W efficiency. OISA reduces power consumption by a factor of 7.9 and 18.4 on average compared with existing electronic in-/near-sensor and ASIC accelerators.

翻译：针对边缘端视觉应用需求，本文首次系统性地探索并提出一种高性能、高能效的光学传感器内加速器架构OISA。该架构利用光子器件的高效特性，以创新的最小转换方式，在低比特宽度神经网络中实现输入帧的粗粒度卷积运算。相较传统云端中心架构及近年提出的边缘加速器，该设计显著降低了数据转换、传输与处理的功耗。基于多种图像数据集的器件到架构级仿真结果表明，OISA在保持可接受的精度的同时，实现了6.68 TOp/s/W的能效比。与现有电子式传感器内/近传感器加速器及ASIC加速器相比，OISA平均功耗降低了7.9倍和18.4倍。