Characterizing two-dimensional quantum materials from optical microscopy images is challenging due to the subtle layer-dependent contrast, limited labeled data, and significant variation across laboratories and imaging setups. Existing vision models struggle in this domain since they lack physical priors and cannot generalize to new materials or hardware conditions. This work presents a new physics-aware multimodal framework that addresses these limitations from both the data and model perspectives. We first present Synthia, a physics-based synthetic data generator that simulates realistic optical responses of quantum material flakes under thin-film interference. Synthia produces diverse and high-quality samples, helping reduce the dependence on expert manual annotation. We introduce QMat-Instruct, the first large-scale instruction dataset for quantum materials, comprising multimodal, physics-informed question-answer pairs designed to teach Multimodal Large Language Models (MLLMs) to understand the appearance and thickness of flakes. Then, we propose Physics-Aware Instruction Tuning (QuPAINT), a multimodal architecture that incorporates a Physics-Informed Attention module to fuse visual embeddings with optical priors, enabling more robust and discriminative flake representations. Finally, we establish QF-Bench, a comprehensive benchmark spanning multiple materials, substrates, and imaging settings, offering standardized protocols for fair and reproducible evaluation.

翻译：从光学显微图像中表征二维量子材料具有挑战性，原因在于层数依赖的对比度差异细微、标记数据有限，以及不同实验室和成像设备间存在显著差异。现有视觉模型在此领域表现不佳，因为它们缺乏物理先验知识，且无法泛化到新材料或新的硬件条件。本研究提出一种新的物理感知多模态框架，从数据和模型两个角度解决这些局限性。我们首先提出了Synthia，一种基于物理的合成数据生成器，用于模拟量子材料薄片在薄膜干涉下的真实光学响应。Synthia能够生成多样且高质量的样本，有助于减少对专家手动标注的依赖。我们引入了QMat-Instruct，这是首个面向量子材料的大规模指令数据集，包含多模态、融合物理知识的问答对，旨在教导多模态大语言模型理解薄片的外观与厚度。接着，我们提出了物理感知指令调优方法QuPAINT，这是一种多模态架构，其包含一个物理信息注意力模块，用于将视觉嵌入与光学先验知识融合，从而实现更鲁棒且更具判别力的薄片表征。最后，我们建立了QF-Bench，这是一个涵盖多种材料、衬底和成像设置的综合基准，为公平且可复现的评估提供了标准化协议。