Purpose: To evaluate the accuracy and reasoning ability of DeepSeek-R1 and three other recently released large language models (LLMs) in bilingual complex ophthalmology cases. Methods: A total of 130 multiple-choice questions (MCQs) related to diagnosis (n = 39) and management (n = 91) were collected from the Chinese ophthalmology senior professional title examination and categorized into six topics. These MCQs were translated into English using DeepSeek-R1. The responses of DeepSeek-R1, Gemini 2.0 Pro, OpenAI o1 and o3-mini were generated under default configurations between February 15 and February 20, 2025. Accuracy was calculated as the proportion of correctly answered questions, with omissions and extra answers considered incorrect. Reasoning ability was evaluated through analyzing reasoning logic and the causes of reasoning error. Results: DeepSeek-R1 demonstrated the highest overall accuracy, achieving 0.862 in Chinese MCQs and 0.808 in English MCQs. Gemini 2.0 Pro, OpenAI o1, and OpenAI o3-mini attained accuracies of 0.715, 0.685, and 0.692 in Chinese MCQs (all P<0.001 compared with DeepSeek-R1), and 0.746 (P=0.115), 0.723 (P=0.027), and 0.577 (P<0.001) in English MCQs, respectively. DeepSeek-R1 achieved the highest accuracy across five topics in both Chinese and English MCQs. It also excelled in management questions conducted in Chinese (all P<0.05). Reasoning ability analysis showed that the four LLMs shared similar reasoning logic. Ignoring key positive history, ignoring key positive signs, misinterpretation medical data, and too aggressive were the most common causes of reasoning errors. Conclusion: DeepSeek-R1 demonstrated superior performance in bilingual complex ophthalmology reasoning tasks than three other state-of-the-art LLMs. While its clinical applicability remains challenging, it shows promise for supporting diagnosis and clinical decision-making.

翻译：目的：评估 DeepSeek-R1 及其他三个近期发布的大语言模型在双语复杂眼科病例中的准确性和推理能力。方法：从中国眼科高级职称考试中收集了总计 130 道与诊断（n = 39）和管理（n = 91）相关的选择题，并将其归类为六个主题。这些选择题使用 DeepSeek-R1 翻译成英文。在 2025 年 2 月 15 日至 2 月 20 日期间，在默认配置下生成了 DeepSeek-R1、Gemini 2.0 Pro、OpenAI o1 和 o3-mini 的响应。准确性计算为正确回答问题的比例，遗漏和额外答案均被视为错误。通过分析推理逻辑和推理错误的原因来评估推理能力。结果：DeepSeek-R1 表现出最高的总体准确性，在中文选择题中达到 0.862，在英文选择题中达到 0.808。Gemini 2.0 Pro、OpenAI o1 和 OpenAI o3-mini 在中文选择题中的准确性分别为 0.715、0.685 和 0.692（与 DeepSeek-R1 相比，所有 P<0.001），在英文选择题中分别为 0.746 (P=0.115)、0.723 (P=0.027) 和 0.577 (P<0.001)。DeepSeek-R1 在中文和英文选择题的五个主题中均取得了最高的准确性。它在中文进行的管理问题中也表现出色（所有 P<0.05）。推理能力分析表明，四个大语言模型具有相似的推理逻辑。忽略关键阳性病史、忽略关键阳性体征、误解医疗数据以及过于激进是最常见的推理错误原因。结论：在双语复杂眼科推理任务中，DeepSeek-R1 的表现优于其他三个最先进的大语言模型。虽然其临床适用性仍面临挑战，但它显示出支持诊断和临床决策的潜力。