Aiding humans with scientific designs is one of the most exciting of artificial intelligence (AI) and machine learning (ML), due to their potential for the discovery of new drugs, design of new materials and chemical compounds, etc. However, scientific design typically requires complex domain knowledge that is not familiar to AI researchers. Further, scientific studies involve professional skills to perform experiments and evaluations. These obstacles prevent AI researchers from developing specialized methods for scientific designs. To take a step towards easy-to-understand and reproducible research of scientific design, we propose a benchmark for the inverse design of nanophotonic devices, which can be verified computationally and accurately. Specifically, we implemented three different nanophotonic design problems, namely a radiative cooler, a selective emitter for thermophotovoltaics, and structural color filters, all of which are different in design parameter spaces, complexity, and design targets. The benchmark environments are implemented with an open-source simulator. We further implemented 10 different inverse design algorithms and compared them in a reproducible and fair framework. The results revealed the strengths and weaknesses of existing methods, which shed light on several future directions for developing more efficient inverse design algorithms. Our benchmark can also serve as the starting point for more challenging scientific design problems. The code of IDToolkit is available at https://github.com/ThyrixYang/IDToolkit.

翻译：辅助人类进行科学设计是人工智能（AI）与机器学习（ML）最令人振奋的应用之一，因其在新药发现、新材料及化学化合物设计等领域具有巨大潜力。然而，科学设计通常需要AI研究者不熟悉的复杂领域知识。此外，科学研究涉及实验操作与评估等专业技能。这些障碍阻碍了AI研究者开发针对科学设计的专门方法。为推进科学设计的易理解与可复现研究，我们提出一个可通过计算精确验证的纳米光子器件逆向设计基准。具体而言，我们实现了三种不同的纳米光子设计问题：辐射冷却器、热光伏选择性发射器以及结构色滤光片，这些问题在设计参数空间、复杂度和设计目标上均存在差异。基准环境基于开源仿真器实现。我们进一步实现了10种不同的逆向设计算法，并在可复现和公平的框架下进行比较。结果揭示了现有方法的优势与不足，为开发更高效的逆向设计算法指明了若干未来方向。该基准亦可作为更具挑战性的科学设计问题的起点。IDToolkit的代码详见 https://github.com/ThyrixYang/IDToolkit。