Recent advancements in LLMs have shown their significant potential in tasks like text summarization and generation. Yet, they often encounter difficulty while solving complex physics problems that require arithmetic calculation and a good understanding of concepts. Moreover, many physics problems include images that contain important details required to understand the problem's context. We propose an LMM-based chatbot to answer multimodal physics MCQs. For domain adaptation, we utilize the MM-PhyQA dataset comprising Indian high school-level multimodal physics problems. To improve the LMM's performance, we experiment with two techniques, RLHF (Reinforcement Learning from Human Feedback) and Image Captioning. In image captioning, we add a detailed explanation of the diagram in each image, minimizing hallucinations and image processing errors. We further explore the integration of Reinforcement Learning from Human Feedback (RLHF) methodology inspired by the ranking approach in RLHF to enhance the human-like problem-solving abilities of the models. The RLHF approach incorporates human feedback into the learning process of LLMs, improving the model's problem-solving skills, truthfulness, and reasoning capabilities, minimizing the hallucinations in the answers, and improving the quality instead of using vanilla-supervised fine-tuned models. We employ the LLaVA open-source model to answer multimodal physics MCQs and compare the performance with and without using RLHF.

翻译：近年来，大语言模型在文本摘要与生成等任务中展现出显著潜力，但在解决需要算术运算及概念理解的复杂物理问题时仍面临困难。此外，许多物理问题包含承载问题语境关键信息的图像。我们提出基于大语言模型（LMM）的对话机器人，用于回答多模态物理选择题。为进行领域适应，我们采用包含印度高中级多模态物理问题的MM-PhyQA数据集。为提升LMM性能，我们实验了两种技术：基于人类反馈的强化学习（RLHF）与图像描述。在图像描述中，我们为每张图像添加详细的图示说明，以减少幻觉与图像处理误差。我们进一步探索融入RLHF方法，借鉴其排序策略增强模型类人解题能力。RLHF方法将人类反馈纳入大语言模型学习过程，提升模型解题能力、真实性与推理能力，减少答案幻觉，并提高质量，优于传统监督微调模型。我们采用LLaVA开源模型回答多模态物理选择题，并对比使用与未使用RLHF的性能表现。