MoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: \textit{``I have felt down, depressed, or hopeless''}. Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools.

翻译：MoodCapture提出了一种创新方法，通过人们在日常活动中由智能手机前置摄像头自动捕获的图像来评估抑郁症。我们收集了177名被诊断为重度抑郁症的参与者在90天内超过12.5万张自然场景照片。这些图像是在参与者回答PHQ-8抑郁调查问题“我感到情绪低落、沮丧或绝望”时自然捕获的。我们的分析探索了重要图像属性，如角度、主色调、位置、物体和光照。研究表明，基于面部标志训练随机森林模型能够有效将样本分类为抑郁或非抑郁状态，并预测原始PHQ-8得分。通过消融研究、特征重要性分析和偏差评估，我们的后验分析提供了多项见解。重要的是，我们评估了用户对使用MoodCapture基于分享照片检测抑郁症的担忧，为隐私顾虑提供了关键见解，这些发现将为未来基于自然场景图像的日常心理健康评估工具设计提供指导。