Using an open problem from the EC 2025 paper "Stable Menus of Public Goods" as a testbed, we conduct experiments to understand the effectiveness of different AI-for-EconCS research workflows. Specifically, we study three questions: Does providing human intuition in the prompt help? Does automated multi-turn interaction help? And, does an LLM outperform a first-year PhD student? Regarding the first two questions, we provide evidence for the following workflow suggestions: (1) prompting with human intuition can encourage the LLM to have better "taste", (2) multi-turn workflows help when the pipeline encourages "ambitious" steps. Regarding the third question, using an unpublished manuscript written by the paper's senior authors prior to collaborating with the first-year PhD student, we compare the effectiveness of the LLM with that of the first-year PhD student, and find that the LLM is slightly less effective.
翻译:以EC 2025论文《公共品稳定菜单》中的一个开放问题作为测试平台,我们通过实验来理解不同AI-for-EconCS研究流程的有效性。具体而言,我们研究三个问题:在提示中提供人类直觉是否有帮助?自动化多轮交互是否有帮助?以及大语言模型是否优于一年级博士生?针对前两个问题,我们为以下流程建议提供了证据:(1)加入人类直觉的提示能促使大语言模型具备更佳的"品味";(2)当流程鼓励"进取性"步骤时,多轮工作流更有助益。针对第三个问题,利用该论文资深作者在与一年级博士生合作前撰写的未发表手稿,我们比较了大语言模型与一年级博士生的效能,发现大语言模型的表现略逊一筹。