Learning is a key motivator behind information search behavior. With the emergence of LLM-based chatbots, students are increasingly turning to these tools as their primary resource for acquiring knowledge. However, the transition from traditional resources like textbooks and web searches raises concerns among educators. They worry that these fully-automated LLMs might lead students to delegate critical steps of search as learning. In this paper, we systematically uncover three main concerns from educators' perspectives. In response to these concerns, we conducted a mixed-methods study with 92 university students to compare three learning sources with different automation levels. Our results show that LLMs support comprehensive understanding of key concepts without promoting passive learning, though their effectiveness in knowledge retention was limited. Additionally, we found that academic performance impacted both learning outcomes and search patterns. Notably, higher-competence learners engaged more deeply with content through reading-intensive behaviors rather than relying on search activities.

翻译：学习是信息搜索行为背后的关键驱动力。随着基于LLM的聊天机器人的出现，学生越来越多地将这些工具作为获取知识的主要资源。然而，从教科书和网络搜索等传统资源向这些工具的转变引起了教育工作者的担忧。他们担心这些全自动的LLM可能导致学生将搜索作为学习的关键步骤委托出去。本文从教育者的视角系统性地揭示了三个主要关切点。针对这些关切，我们对92名大学生进行了混合方法研究，比较了三种自动化程度不同的学习资源。研究结果表明，LLM能够支持对关键概念的全面理解，且不会助长被动学习，但其在知识留存方面的效果有限。此外，我们发现学业成绩既影响学习成果，也影响搜索模式。值得注意的是，高能力学习者通过密集阅读行为而非依赖搜索活动，与学习内容进行了更深入的互动。