We focus on addressing the challenges in responsible beauty product recommendation, particularly when it involves comparing the product's color with a person's skin tone, such as for foundation and concealer products. To make accurate recommendations, it is crucial to infer both the product attributes and the product specific facial features such as skin conditions or tone. However, while many product photos are taken under good light conditions, face photos are taken from a wide range of conditions. The features extracted using the photos from ill-illuminated environment can be highly misleading or even be incompatible to be compared with the product attributes. Hence bad illumination condition can severely degrade quality of the recommendation. We introduce a machine learning framework for illumination assessment which classifies images into having either good or bad illumination condition. We then build an automatic user guidance tool which informs a user holding their camera if their illumination condition is good or bad. This way, the user is provided with rapid feedback and can interactively control how the photo is taken for their recommendation. Only a few studies are dedicated to this problem, mostly due to the lack of dataset that is large, labeled, and diverse both in terms of skin tones and light patterns. Lack of such dataset leads to neglecting skin tone diversity. Therefore, We begin by constructing a diverse synthetic dataset that simulates various skin tones and light patterns in addition to an existing facial image dataset. Next, we train a Convolutional Neural Network (CNN) for illumination assessment that outperforms the existing solutions using the synthetic dataset. Finally, we analyze how the our work improves the shade recommendation for various foundation products.

翻译：我们聚焦于解决负责任的美容产品推荐中的挑战，特别是在将产品颜色与个人肤色进行比对时，例如粉底和遮瑕膏产品。为了做出精准推荐，必须同时推断产品属性以及产品相关的面部特征，如皮肤状况或色调。然而，尽管许多产品照片在良好光照条件下拍摄，面部照片却取自各种光照条件。从光照不佳环境下拍摄的照片中提取的特征可能极具误导性，甚至无法与产品属性进行有效比对。因此，不良光照条件会严重降低推荐质量。我们引入了一个基于机器学习的光照评估框架，该框架将图像分类为光照良好或光照不佳两类。随后，我们构建了一个自动用户引导工具，当用户持握相机时，该工具会告知其当前光照条件是否良好。通过这种方式，用户能获得即时反馈，并能够交互式地控制如何拍摄用于推荐的图像。目前仅有少量研究致力于解决此问题，主要原因在于缺乏规模庞大、标注完善且涵盖肤色与光照模式多样性的数据集。此类数据集的缺失导致了肤色多样性的忽视。因此，我们首先构建了一个多样化的合成数据集，该数据集在模拟多种肤色与光照模式的同时，还结合了现有的面部图像数据集。接着，我们训练了一个用于光照评估的卷积神经网络（CNN），该网络利用合成数据集在性能上超越了现有解决方案。最后，我们分析了本文工作如何提升各类粉底产品的色号推荐效果。