This study explores the problem solving capabilities of ChatGPT and its prospective applications in standardized test preparation, focusing on the GRE quantitative exam. Prior research has shown great potential for the utilization of ChatGPT for academic purposes in revolutionizing the approach to studying across various disciplines. We investigate how ChatGPT performs across various question types in the GRE quantitative domain, and how modifying question prompts impacts its accuracy. More specifically this study addressed two research questions: 1. How does ChatGPT perform in answering GRE-based quantitative questions across various content areas? 2. How does the accuracy of ChatGPT vary with modifying the question prompts? The dataset consisting of 100 randomly selected GRE quantitative questions was collected from the ETS official guide to GRE test preparation. We used quantitative evaluation to answer our first research question, and t-test to examine the statistical association between prompt modification and ChatGPT's accuracy. Results show a statistical improvement in the ChatGPT's accuracy after applying instruction priming and contextual prompts to the original questions. ChatGPT showed 84% accuracy with the modified prompts compared to 69% with the original data. The study discusses the areas where ChatGPT struggled with certain questions and how modifications can be helpful for preparing for standardized tests like GRE and provides future directions for prompt modifications.

翻译：本研究探讨了ChatGPT的问题解决能力及其在标准化考试备考中的潜在应用，重点关注GRE数学科目考试。已有研究表明，ChatGPT在学术领域的应用具有巨大潜力，能够革新各学科的学习方式。我们考察了ChatGPT在GRE数学各题型中的表现，以及问题提示的修改对其准确率的影响。具体而言，本研究探讨两个研究问题：1. ChatGPT在回答不同内容领域的GRE数学问题时的表现如何？2. 修改问题提示后ChatGPT的准确率如何变化？数据集包含从ETS官方GRE备考指南中随机选取的100道数学问题。我们采用定量评估回答第一个研究问题，通过t检验检验提示修改与ChatGPT准确率之间的统计关联。结果表明，对原始问题施加指令启动和上下文提示后，ChatGPT的准确率出现统计上的显著提升。使用修改后的提示时，ChatGPT准确率达到84%，而原始数据仅为69%。本研究讨论了ChatGPT在特定问题上存在的困难，以及如何通过提示修改为GRE等标准化考试备考提供帮助，并为提示修改的未来方向提出了建议。