Evaluations of large language models (LLMs) in scientific information seeking tasks have become increasingly use-centric, such as conducting live or multi-turn evaluations with real users. These evaluations still assume a single, static chat interface, but as models are integrated into new interfaces, evaluations must shift to incorporate interface-specific criteria. We propose a new evaluation framework based on a formative study with $16$ participants that tests models' ability to generate multiple responses to one query that differ along an interpretable axis of language (language complexity), inspired by direct manipulation interfaces from human-centered design literature. We evaluate GPT-5.1, GPT-5 mini, Claude Sonnet 4.5 + Thinking, and DeepSeek-V3.1 by generating 5 responses at different levels of language complexity for $98$ scientific queries. While models vary complexity across responses, most changes remain inconsistent, with the best performing model (Claude Sonnet 4.5) only shifting reliable complexity measures in the correct direction $46\%$ of the time. Our findings hold with increased sample size and alternative complexity levels.

翻译：针对科学信息检索任务的大型语言模型（LLM）评估正日益转向以用户为中心，例如结合真实用户进行实时或多轮评估。然而，这类评估仍假设单一、静态的聊天界面。随着模型被整合到新型界面中，评估必须纳入界面特定标准。我们提出一种新的评估框架，该框架基于一项包含16名参与者的形成性研究，受以人为本设计文献中的直接操纵界面启发，测试模型针对同一查询生成多条回应的能力，这些回应沿语言可解释轴（语言复杂度）呈现差异。我们评估了GPT-5.1、GPT-5 mini、Claude Sonnet 4.5 + Thinking和DeepSeek-V3.1，针对98个科学查询生成了5个不同语言复杂度层级的回应。尽管模型在回应的复杂度上有所变化，但多数变化仍不一致，表现最佳的模型（Claude Sonnet 4.5）仅能在46%的情况下将可靠复杂度指标朝正确方向调整。我们的发现在增加样本量和采用替代复杂度层级时依然成立。