To lower the barrier to diffractive optical neural networks (DONNs) design, exploration, and deployment, we propose LightRidge, the first end-to-end optical ML compilation framework, which consists of (1) precise and differentiable optical physics kernels that enable complete explorations of DONNs architectures, (2) optical physics computation kernel acceleration that significantly reduces the runtime cost in training, emulation, and deployment of DONNs, and (3) versatile and flexible optical system modeling and user-friendly domain-specific-language (DSL). As a result, LightRidge framework enables efficient end-to-end design and deployment of DONNs, and significantly reduces the efforts for programming, hardware-software codesign, and chip integration. Our results are experimentally conducted with physical optical systems, where we demonstrate: (1) the optical physics kernels precisely correlated to low-level physics and systems, (2) significant speedups in runtime with physics-aware emulation workloads compared to the state-of-the-art commercial system, (3) effective architectural design space exploration verified by the hardware prototype and on-chip integration case study, and (4) novel DONN design principles including successful demonstrations of advanced image classification and image segmentation task using DONNs architecture and topology.
翻译:为降低衍射光神经网络(DONN)设计、探索与部署的门槛,我们提出LightRidge——首个端到端光学机器学习编译框架。该框架包含:(1)精确可微的光学物理内核,支持DONN架构的全面探索;(2)光学物理计算内核加速技术,显著降低DONN在训练、仿真与部署中的运行时成本;(3)通用灵活的光学系统建模与用户友好的领域特定语言(DSL)。基于此,LightRidge框架实现了DONN的高效端到端设计与部署,并大幅降低了编程、软硬件协同设计与芯片集成的工作量。我们通过物理光学系统的实验验证表明:(1)光学物理内核与底层物理系统和硬件精确关联;(2)相较于当前最先进的商用系统,物理感知仿真负载的运行时性能显著提升;(3)通过硬件原型与片上集成案例研究验证有效的架构设计空间探索能力;(4)提出新型DONN设计准则,成功实现基于DONN架构与拓扑的先进图像分类与图像分割任务。