We present MÖVE (Modelle für die Öffentliche Verwaltung Evaluieren), a holistic benchmark for evaluating large language models (LLMs) in the context of the German public sector. While LLMs are increasingly adopted in public administration, model selection remains largely ad hoc, and existing benchmarks offer limited guidance: they are predominantly English-centric, US-centric in content, and focus exclusively on task performance. MÖVE addresses these gaps by evaluating 39 models across two complementary dimensions. Performance criteria cover summarization, question answering, and topic extraction. Governance criteria assess hallucination tendencies, energy consumption, provider transparency, and alignment with German constitutional values and knowledge about positions by German political parties. In total, we utilize ten German-language datasets, including gold- and silverstandard datasets that we constructed to reflect public-administration domains. We employ a multi-metric evaluation strategy combining classical NLP metrics, embedding-based methods, and LLM-as-a-judge approaches. Our results show that no single model dominates across all criteria: top performers differ between tasks, and model size alone is a poor predictor of quality. We further evaluate the benchmark itself, analyzing its statistical precision, LLM judge reliability, the impact of our private datasets on model rankings, the sensitivity of our results to prompt formulation, and the validity of our energy consumption estimates. MÖVE is designed as a living benchmark under active development; results are publicly available at https://moeve.bundesdruckerei.de/.

翻译：我们提出MÖVE（Modelle für die Öffentliche Verwaltung Evaluieren），这是一个用于评估大型语言模型（LLM）在德国公共部门应用中的全面基准测试。尽管LLM在公共行政领域的应用日益广泛，但模型选择仍主要依赖临时决策，现有基准测试的指导作用有限：它们大多以英语为中心、内容偏重美国语境，且仅关注任务性能。MÖVE通过评估39个模型在两个互补维度上的表现来弥补这些不足。性能维度涵盖摘要生成、问答和主题提取。治理维度评估幻觉倾向、能耗、供应商透明度、与德国宪法价值观的契合度，以及对德国政党立场的认知。我们总共使用了十个德语数据集，包括我们自行构建的反映公共行政领域的黄金和白银标准数据集。我们采用多指标评估策略，综合运用经典自然语言处理指标、基于嵌入的方法以及LLM-as-a-judge方法。结果表明，没有单一模型在所有标准上占优：不同任务的最佳表现者各异，且模型规模本身并非质量的可靠预测指标。我们还对基准测试本身进行了评估，分析了其统计精度、LLM评判者可靠性、私有数据集对模型排名的影响、结果对提示表述的敏感性，以及能耗估算的有效性。MÖVE设计为一个持续开发中的动态基准；结果公开可见于https://moeve.bundesdruckerei.de/。