Intrinsic image decomposition and inverse rendering are long-standing problems in computer vision. To evaluate albedo recovery, most algorithms report their quantitative performance with a mean Weighted Human Disagreement Rate (WHDR) metric on the IIW dataset. However, WHDR focuses only on relative albedo values and often fails to capture overall quality of the albedo. In order to comprehensively evaluate albedo, we collect a new dataset, Measured Albedo in the Wild (MAW), and propose three new metrics that complement WHDR: intensity, chromaticity and texture metrics. We show that existing algorithms often improve WHDR metric but perform poorly on other metrics. We then finetune different algorithms on our MAW dataset to significantly improve the quality of the reconstructed albedo both quantitatively and qualitatively. Since the proposed intensity, chromaticity, and texture metrics and the WHDR are all complementary we further introduce a relative performance measure that captures average performance. By analysing existing algorithms we show that there is significant room for improvement. Our dataset and evaluation metrics will enable researchers to develop algorithms that improve albedo reconstruction. Code and Data available at: https://measuredalbedo.github.io/

翻译：本征图像分解与逆渲染是计算机视觉领域的长期难题。为评估反照率恢复效果，现有算法大多基于IIW数据集采用平均加权人类分歧率（WHDR）指标报告定量性能。然而，WHDR仅关注相对反照率值，往往无法捕捉反照率的整体质量。为全面评估反照率，我们收集了新数据集——野外实测反照率（MAW），并提出三项互补WHDR的新指标：强度指标、色度指标和纹理指标。研究表明，现有算法虽能提升WHDR指标，但在其他指标上表现不佳。通过在MAW数据集上微调不同算法，我们显著提高了重建反照率的定量与定性质量。鉴于所提出的强度、色度、纹理指标与WHDR具有互补性，我们进一步引入可表征平均性能的相对性能度量指标。通过分析现有算法，我们发现其仍存在显著改进空间。本数据集与评估指标将推动研究者开发改进反照率重建的算法。代码与数据获取地址：https://measuredalbedo.github.io/