Aiding humans with scientific designs is one of the most exciting of artificial intelligence (AI) and machine learning (ML), due to their potential for the discovery of new drugs, design of new materials and chemical compounds, etc. However, scientific design typically requires complex domain knowledge that is not familiar to AI researchers. Further, scientific studies involve professional skills to perform experiments and evaluations. These obstacles prevent AI researchers from developing specialized methods for scientific designs. To take a step towards easy-to-understand and reproducible research of scientific design, we propose a benchmark for the inverse design of nanophotonic devices, which can be verified computationally and accurately. Specifically, we implemented three different nanophotonic design problems, namely a radiative cooler, a selective emitter for thermophotovoltaics, and structural color filters, all of which are different in design parameter spaces, complexity, and design targets. The benchmark environments are implemented with an open-source simulator. We further implemented 10 different inverse design algorithms and compared them in a reproducible and fair framework. The results revealed the strengths and weaknesses of existing methods, which shed light on several future directions for developing more efficient inverse design algorithms. Our benchmark can also serve as the starting point for more challenging scientific design problems. The code of IDToolkit is available at https://github.com/ThyrixYang/IDToolkit.

翻译：在科学设计中辅助人类是人工智能和机器学习最令人振奋的应用之一，因其具有发现新药物、设计新材料和化合物等潜力。然而，科学设计通常需要复杂的领域知识，而这对于AI研究者而言并不熟悉。此外，科学研究涉及专业实验与评估技能。这些障碍阻碍了AI研究者开发针对科学设计的专门方法。为迈向易于理解和可复现的科学设计研究，我们提出了一个可通过计算精确验证的纳米光子器件逆向设计基准。具体而言，我们实现了三个不同的纳米光子设计问题：辐射冷却器、热光伏选择性发射器以及结构彩色滤光片，这三个问题在设计参数空间、复杂度和设计目标上均存在差异。基准环境采用开源模拟器实现。我们进一步实现了10种不同的逆向设计算法，并在可复现且公平的框架下进行了比较。结果揭示了现有方法的优势与不足，为开发更高效逆向设计算法的未来方向提供了启示。我们的基准也可作为更具挑战性科学设计问题的起点。IDToolkit的代码可在https://github.com/ThyrixYang/IDToolkit获取。