Skin and subcutaneous diseases rank high among the leading contributors to the global burden of nonfatal diseases, impacting a considerable portion of the population. Nonetheless, the field of dermatology diagnosis faces three significant hurdles. Firstly, there is a shortage of dermatologists accessible to diagnose patients, particularly in rural regions. Secondly, accurately interpreting skin disease images poses a considerable challenge. Lastly, generating patient-friendly diagnostic reports is usually a time-consuming and labor-intensive task for dermatologists. To tackle these challenges, we present SkinGPT-4, which is the world's first interactive dermatology diagnostic system powered by an advanced visual large language model. SkinGPT-4 leverages a fine-tuned version of MiniGPT-4, trained on an extensive collection of skin disease images (comprising 52,929 publicly available and proprietary images) along with clinical concepts and doctors' notes. We designed a two-step training process to allow SkinGPT to express medical features in skin disease images with natural language and make accurate diagnoses of the types of skin diseases. With SkinGPT-4, users could upload their own skin photos for diagnosis, and the system could autonomously evaluate the images, identifies the characteristics and categories of the skin conditions, performs in-depth analysis, and provides interactive treatment recommendations. Meanwhile, SkinGPT-4's local deployment capability and commitment to user privacy also render it an appealing choice for patients in search of a dependable and precise diagnosis of their skin ailments. To demonstrate the robustness of SkinGPT-4, we conducted quantitative evaluations on 150 real-life cases, which were independently reviewed by certified dermatologists, and showed that SkinGPT-4 could provide accurate diagnoses of skin diseases.

翻译：皮肤及皮下疾病是非致命性疾病的全球主要负担之一，影响着相当大比例的人口。然而，皮肤病诊断领域面临三大挑战：首先，尤其在偏远地区，可供患者就诊的皮肤科医生严重短缺；其次，准确解读皮肤疾病图像极具挑战性；最后，生成患者友好的诊断报告对皮肤科医生而言通常是一项耗时费力的任务。为应对这些挑战，我们提出SkinGPT-4——这是全球首个由先进视觉大语言模型驱动的交互式皮肤病诊断系统。SkinGPT-4基于微调版MiniGPT-4，在包含52,929张公开和专有皮肤疾病图像的大规模数据集上训练，并融合了临床概念与医生笔记。我们设计了两阶段训练流程，使SkinGPT能够通过自然语言表达皮肤疾病图像的医学特征，并准确诊断皮肤疾病类型。借助SkinGPT-4，用户可上传自身皮肤照片进行诊断，系统能自主评估图像、识别皮肤状况的特征与类别、进行深度分析，并提供交互式治疗建议。同时，SkinGPT-4的本地部署能力及对用户隐私的承诺，使其成为寻求可靠精准皮肤疾病诊断患者的理想选择。为验证SkinGPT-4的鲁棒性，我们基于150例真实病例进行了定量评估（由持证皮肤科医生独立审核），结果表明SkinGPT-4能够提供准确的皮肤疾病诊断。