Incorporation of physical information in machine learning frameworks are opening and transforming many application domains. Here the learning process is augmented through the induction of fundamental knowledge and governing physical laws. In this work we explore their utility for computer vision tasks in interpreting and understanding visual data. We present a systematic literature review of formulation and approaches to computer vision tasks guided by physical laws, known as physics-informed computer vision. We begin by decomposing the popular computer vision pipeline into a taxonomy of stages and investigate approaches to incorporate governing physical equations in each stage. Existing approaches in each task are analyzed with regard to what governing physical processes are modeled for integration and how they are formulated to be incorporated, i.e. modify data (observation bias), modify networks (inductive bias), and modify losses (learning bias) to include physical rules. The taxonomy offers a unified view of the application of the physics-informed capability, highlighting where physics-informed machine learning has been conducted and where the gaps and opportunities are. Finally, we highlight open problems and challenges to inform future research avenues. While still in its early days, the study of physics-informed computer vision has the promise to develop better computer vision models that can improve physical plausibility, accuracy, data efficiency and generalization in increasingly realistic applications.

翻译：在机器学习框架中融入物理信息正在开启并改变众多应用领域。通过引入基础知识和支配性物理定律，学习过程得以增强。本文探讨了这些方法在计算机视觉任务中用于解释和理解视觉数据的效用。我们系统综述了受物理定律指导的计算机视觉任务（即物理启发的计算机视觉）的公式化方法与实现途径。首先，将主流计算机视觉流程分解为阶段分类法，研究在每个阶段融入支配性物理方程的方法。针对各任务现有方法，分析其建模了哪些支配性物理过程以实现融合，以及如何通过公式化手段加以整合——即修改数据（观测偏差）、修改网络（归纳偏差）和修改损失函数（学习偏差）以纳入物理规则。该分类法提供了物理启发能力的统一视角，揭示了物理启发机器学习已开展的领域以及存在的空白与机遇。最后，我们强调当前开放问题与挑战，为未来研究方向提供参考。尽管仍处于早期阶段，物理启发的计算机视觉研究有望开发出更优的计算机视觉模型，在日益真实的应用场景中提升物理合理性、准确性、数据效率与泛化能力。