Analog computing using non-volatile memristors has emerged as a promising solution for energy-efficient deep learning. New materials, like perovskites-based memristors are recently attractive due to their cost-effectiveness, energy efficiency and flexibility. Yet, challenges in material diversity and immature fabrications require extensive experimentation for device development. Moreover, significant non-idealities in these memristors often impede them for computing. Here, we propose a synergistic methodology to concurrently optimize perovskite memristor fabrication and develop robust analog DNNs that effectively address the inherent non-idealities of these memristors. Employing Bayesian optimization (BO) with a focus on usability, we efficiently identify optimal materials and fabrication conditions for perovskite memristors. Meanwhile, we developed "BayesMulti", a DNN training strategy utilizing BO-guided noise injection to improve the resistance of analog DNNs to memristor imperfections. Our approach theoretically ensures that within a certain range of parameter perturbations due to memristor non-idealities, the prediction outcomes remain consistent. Our integrated approach enables use of analog computing in much deeper and wider networks, which significantly outperforms existing methods in diverse tasks like image classification, autonomous driving, species identification, and large vision-language models, achieving up to 100-fold improvements. We further validate our methodology on a 10$\times$10 optimized perovskite memristor crossbar, demonstrating high accuracy in a classification task and low energy consumption. This study offers a versatile solution for efficient optimization of various analog computing systems, encompassing both devices and algorithms.
翻译:利用非易失性忆阻器的模拟计算已成为实现高能效深度学习的一种有前景的解决方案。基于钙钛矿的新型忆阻器材料,由于其成本效益高、能效优异及灵活性好,近来备受关注。然而,材料多样性方面的挑战以及不成熟的制造工艺,使得器件开发需要进行大量的实验。此外,这些忆阻器中显著的非理想特性常常阻碍其用于计算。在此,我们提出一种协同优化方法,旨在同步优化钙钛矿忆阻器的制造工艺,并开发能够有效应对这些忆阻器固有非理想特性的鲁棒模拟深度神经网络。我们采用以可用性为核心的贝叶斯优化方法,高效地确定了钙钛矿忆阻器的最佳材料与制造条件。同时,我们开发了"BayesMulti"——一种利用贝叶斯优化引导的噪声注入的深度神经网络训练策略,以增强模拟深度神经网络对忆阻器缺陷的抵抗力。我们的方法从理论上保证了,在因忆阻器非理想特性导致的参数扰动的一定范围内,预测结果保持一致。我们这种集成的方案使得模拟计算能够应用于更深更广的网络中,在图像分类、自动驾驶、物种识别以及大型视觉-语言模型等多种任务上显著优于现有方法,实现了高达100倍的性能提升。我们进一步在一个10$\times$10的优化钙钛矿忆阻器交叉阵列上验证了我们的方法,在分类任务中展示了高精度和低能耗。本研究为高效优化涵盖器件与算法的各类模拟计算系统提供了一个通用解决方案。