Authoring high-quality digital materials is key to realism in 3D rendering. Previous generative models for materials have been trained exclusively on synthetic data; such data is limited in availability and has a visual gap to real materials. We circumvent this limitation by proposing PhotoMat: the first material generator trained exclusively on real photos of material samples captured using a cell phone camera with flash. Supervision on individual material maps is not available in this setting. Instead, we train a generator for a neural material representation that is rendered with a learned relighting module to create arbitrarily lit RGB images; these are compared against real photos using a discriminator. We then train a material maps estimator to decode material reflectance properties from the neural material representation. We train PhotoMat with a new dataset of 12,000 material photos captured with handheld phone cameras under flash lighting. We demonstrate that our generated materials have better visual quality than previous material generators trained on synthetic data. Moreover, we can fit analytical material models to closely match these generated neural materials, thus allowing for further editing and use in 3D rendering.

翻译：高质量数字材质的创作是三维渲染真实感的关键。以往的材质生成模型仅在合成数据上训练；此类数据可用性有限，且与真实材质存在视觉差异。我们通过提出PhotoMat突破了这一局限：这是首个仅利用手机摄像头在闪光灯下拍摄的真实材质样本照片训练的材质生成器。在此设置下，无法获得单个材质贴图的监督信号。取而代之的是，我们训练一个生成器以产生神经材质表示，该表示通过一个学习到的重光照模块进行渲染，生成任意光照条件下的RGB图像；这些图像通过判别器与真实照片进行比较。随后，我们训练一个材质贴图估计器，从神经材质表示中解码材质的反射属性。我们使用一个包含12,000张手持手机在闪光条件下拍摄的材质照片的新数据集训练PhotoMat。实验证明，我们生成的材质在视觉质量上优于以往基于合成数据训练的材质生成器。此外，我们能够拟合解析材质模型以紧密匹配这些生成的神经材质，从而支持进一步编辑并在三维渲染中使用。