Despite their promise to facilitate new scientific discoveries, the opaqueness of neural networks presents a challenge in interpreting the logic behind their findings. Here, we use a eXplainable-AI (XAI) technique called $inception$ or $deep$ $dreaming$, which has been invented in machine learning for computer vision. We use this techniques to explore what neural networks learn about quantum optics experiments. Our story begins by training a deep neural networks on the properties of quantum systems. Once trained, we "invert" the neural network -- effectively asking how it imagines a quantum system with a specific property, and how it would continuously modify the quantum system to change a property. We find that the network can shift the initial distribution of properties of the quantum system, and we can conceptualize the learned strategies of the neural network. Interestingly, we find that, in the first layers, the neural network identifies simple properties, while in the deeper ones, it can identify complex quantum structures and even quantum entanglement. This is in reminiscence of long-understood properties known in computer vision, which we now identify in a complex natural science task. Our approach could be useful in a more interpretable way to develop new advanced AI-based scientific discovery techniques in quantum physics.

翻译：尽管神经网络有望促进新的科学发现，但其不透明性对解释其发现背后的逻辑提出了挑战。在此，我们使用一种名为“inception”或“deep dreaming”的可解释人工智能（XAI）技术，该技术源于机器学习领域的计算机视觉。我们利用这一技术探索神经网络对量子光学实验的学习内容。我们的研究从训练一个深度神经网络学习量子系统的属性开始。训练完成后，我们“反转”该神经网络——实质上是在询问它如何想象一个具有特定属性的量子系统，以及它将如何持续修改该量子系统以改变这一属性。我们发现，网络能够改变量子系统属性的初始分布，并且我们可以概念化神经网络学到的策略。有趣的是，我们注意到，在较浅的层级中，神经网络识别的是简单属性，而在较深的层级中，它能够识别复杂的量子结构甚至量子纠缠。这让人联想到计算机视觉中早已被理解的特性，而我们现在在复杂的自然科学任务中识别到了这一点。我们的方法可能以更可解释的方式，有助于开发量子物理学中基于人工智能的新型先进科学发现技术。