N-of-1 trials aim to estimate treatment effects on the individual level and can be applied to personalize a wide range of physical and digital interventions in mHealth. In this study, we propose and apply a framework for multimodal N-of-1 trials in order to allow the inclusion of health outcomes assessed through images, audio or videos. We illustrate the framework in a series of N-of-1 trials that investigate the effect of acne creams on acne severity assessed through pictures. For the analysis, we compare an expert-based manual labelling approach with different deep learning-based pipelines where in a first step, we train and fine-tune convolutional neural networks (CNN) on the images. Then, we use a linear mixed model on the scores obtained in the first step in order to test the effectiveness of the treatment. The results show that the CNN-based test on the images provides a similar conclusion as tests based on manual expert ratings of the images, and identifies a treatment effect in one individual. This illustrates that multimodal N-of-1 trials can provide a powerful way to identify individual treatment effects and can enable large-scale studies of a large variety of health outcomes that can be actively and passively assessed using technological advances in order to personalized health interventions.

翻译：N-of-1试验旨在个体层面评估治疗效果，并可应用于移动健康领域中的物理及数字干预措施的个性化调整。本研究提出并应用了多模态N-of-1试验框架，以纳入通过图像、音频或视频评估的健康结局。我们通过一系列N-of-1试验对该框架进行验证，探究祛痘霜对基于图片评估的痤疮严重程度的影响。在分析中，我们比较了基于专家手动标注的方法与基于深度学习的多种分析流程：第一步，对卷积神经网络（CNN）进行训练与微调，将其应用于图像分析；第二步，基于第一步获得的评分构建线性混合模型，以检验治疗有效性。结果表明，基于CNN的图像分析所得结论与基于专家手动评分的检验结果一致，并识别出单一个体的治疗效果。这证明多模态N-of-1试验可提供识别个体治疗效应的有效途径，并能借助技术进展主动及被动评估多种健康结局，从而为大规模个性化健康干预研究提供支持。