One hypothesis for the success of deep neural networks (DNNs) is that they are highly expressive, which enables them to be applied to many problems, and they have a strong inductive bias towards solutions that are simple, known as simplicity bias, which allows them to generalise well on unseen data because most real-world data is structured (i.e. simple). In this work, we explore the inductive bias and expressivity of quantum neural networks (QNNs), which gives us a way to compare their performance to those of DNNs. Our results show that it is possible to have simplicity bias with certain QNNs, but we prove that this type of QNN limits the expressivity of the QNN. We also show that it is possible to have QNNs with high expressivity, but they either have no inductive bias or a poor inductive bias and result in a worse generalisation performance compared to DNNs. We demonstrate that an artificial (restricted) inductive bias can be produced by intentionally restricting the expressivity of a QNN. Our results suggest a bias-expressivity tradeoff. Our conclusion is that the QNNs we studied can not generally offer an advantage over DNNs, because these QNNs either have a poor inductive bias or poor expressivity compared to DNNs.

翻译：深度神经网络（DNNs）成功的一种假设是：它们具有高度表达能力，可应用于众多问题；同时它们对简洁解具有强烈的归纳偏好，即简洁性偏好，这使其在未见数据上能良好泛化，因为大多数现实世界数据是结构化（即简洁）的。本研究探讨了量子神经网络（QNNs）的归纳偏好与表达能力，从而为比较其与DNNs的性能提供了依据。结果表明，特定类型的QNNs可能具备简洁性偏好，但我们证明此类QNN会限制其表达能力。同时，我们发现可能存在高表达能力的QNNs，但它们要么缺乏归纳偏好，要么归纳偏好较差，导致其泛化性能不及DNNs。我们证明，通过刻意限制QNN的表达能力，可以人为（受限地）产生归纳偏好。这些结果揭示了偏好与表达能力之间的权衡。我们的结论是：所研究的QNNs通常无法超越DNNs，因为相较于DNNs，这些QNNs要么归纳偏好较差，要么表达能力不足。