As the large language model (LLM) ecosystem expands, individual models exhibit varying capabilities across queries, benchmarks, and domains, motivating the development of LLM routing. While prior work has largely focused on router mechanism design, LLM profiles, which capture model capabilities, remain underexplored. In this work, we ask: How does LLM profile design affect routing performance across different routers? Addressing this question helps clarify the role of profiles in routing, disentangle profile design from router design, and enable fairer comparison and more principled development of routing systems. To this end, we view LLM profiling as a structured information integration problem over heterogeneous interaction histories. We develop a general design space of LLM profiles, named RouteProfile, along four key dimensions: organizational form, representation type, aggregation depth, and learning configuration. Through systematic evaluation across three representative routers under both standard and new-LLM generalization settings, we show that: (1) structured profiles consistently outperform flat ones; (2) query-level signals are more reliable than coarse domain-level signals; and (3) generalization to newly introduced models benefits most from structured profiles under trainable configurations. Overall, our work highlights LLM profile design as an important direction for future routing research.
翻译:随着大语言模型生态系统的扩展,不同模型在查询、基准测试和领域上展现出差异化的能力,这推动了路由机制的发展。尽管先前研究主要聚焦于路由器的机制设计,但用于刻画模型能力的“模型画像”仍未被充分探索。本文提出核心问题:模型画像的设计如何影响不同路由器的路由性能?回答该问题有助于阐明模型画像在路由中的作用、解耦画像设计与路由器设计,并推动路由系统更公平的比较与更规范化的开发。为此,我们将模型画像构建视为一个在异构交互历史上进行的结构化信息整合问题,并沿四个关键维度构建了通用模型画像设计空间——RouteProfile:组织形态、表征类型、聚合深度与学习配置。通过在三种代表性路由器下进行标准设置与新模型泛化设置的系统性评估,我们得出以下结论:(1)结构化画像始终优于扁平化画像;(2)查询级信号比粗粒度的领域级信号更可靠;(3)在新引入模型的泛化场景中,可训练配置下的结构化画像受益最为显著。总体而言,本工作揭示了模型画像设计是未来路由研究的重要方向。