To mitigate the trade-offs between performance and costs, LLM providers route user tasks to different models based on task difficulty and latency. We study the effect of LLM routing with respect to user behavior. We propose a game between an LLM provider with two models (standard and reasoning) and a user who can re-prompt or abandon tasks if the routed model cannot solve them. The user's goal is to maximize their utility minus the delay from using the model, while the provider minimizes the cost of servicing the user. We solve this Stackelberg game by fully characterizing the user best response and simplifying the provider problem. We observe that in nearly all cases, the optimal routing policy involves a static policy with no cascading that depends on the expected utility of the models to the user. Furthermore, we reveal a misalignment gap between the provider-optimal and user-preferred routes when the user's and provider's rankings of the models with respect to utility and cost differ. Finally, we demonstrate conditions for extreme misalignment where providers are incentivized to throttle the latency of the models to minimize their costs, consequently depressing user utility. The results yield simple threshold rules for single-provider, single-user interactions and clarify when routing, cascading, and throttling help or harm.

翻译：为缓解性能与成本之间的权衡，大型语言模型（LLM）提供商会根据任务难度与延迟将用户任务路由至不同模型。本文研究LLM路由对用户行为的影响。我们构建了一个博弈模型，参与者包括一个拥有两种模型（标准型与推理型）的LLM提供商，以及一个在路由模型无法解决问题时可选择重新提示或放弃任务的用户。用户的目标是最大化其效用减去使用模型产生的延迟，而提供商则致力于最小化服务用户的成本。我们通过完整刻画用户最优响应并简化提供商问题来求解这一斯塔克尔伯格博弈。研究发现，在几乎所有情况下，最优路由策略均为静态策略且不涉及级联，该策略取决于模型对用户的期望效用。此外，当用户与提供商对模型在效用和成本方面的排序存在差异时，我们揭示了提供商最优路由与用户偏好路由之间的错配缺口。最后，我们证明了极端错配情形的存在条件：在此条件下提供商有意通过限制模型延迟来最小化成本，从而导致用户效用降低。研究结果为单提供商-单用户交互场景提供了简洁的阈值规则，并阐明了路由、级联与延迟控制何时产生积极或消极影响。