This paper introduces 6G-Bench, an open benchmark for evaluating semantic communication and network-level reasoning in AI-native 6G networks. 6G-Bench defines a taxonomy of 30 decision-making tasks (T1--T30) extracted from ongoing 6G and AI-agent standardization activities in 3GPP, IETF, ETSI, ITU-T, and the O-RAN Alliance, and organizes them into five standardization-aligned capability categories. Starting from 113,475 scenarios, we generate a balanced pool of 10,000 very-hard multiple-choice questions using task-conditioned prompts that enforce multi-step quantitative reasoning under uncertainty and worst-case regret minimization over multi-turn horizons. After automated filtering and expert human validation, 3,722 questions are retained as a high-confidence evaluation set, while the full pool is released to support training and fine-tuning of 6G-specialized models. Using 6G-Bench, we evaluate 22 foundation models spanning dense and mixture-of-experts architectures, short- and long-context designs (up to 1M tokens), and both open-weight and proprietary systems. Across models, deterministic single-shot accuracy (pass@1) spans a wide range from 0.22 to 0.82, highlighting substantial variation in semantic reasoning capability. Leading models achieve intent and policy reasoning accuracy in the range 0.87--0.89, while selective robustness analysis on reasoning-intensive tasks shows pass@5 values ranging from 0.20 to 0.91. To support open science and reproducibility, we release the 6G-Bench dataset on GitHub: https://github.com/maferrag/6G-Bench

翻译：本文介绍6G-Bench，一个用于评估AI原生6G网络中语义通信与网络级推理的开放基准。6G-Bench从3GPP、IETF、ETSI、ITU-T和O-RAN联盟正在进行的6G与AI智能体标准化活动中提取出30个决策任务（T1–T30）的分类体系，并将其组织为五个与标准化对齐的能力类别。从113,475个初始场景出发，我们通过任务条件提示生成包含10,000道极难多选题的平衡题库，这些提示强制要求模型在不确定性下进行多步定量推理，并在多轮时间跨度上实现最坏情况后悔最小化。经过自动筛选和专家人工验证，3,722道题目被保留为高置信度评估集，同时完整题库已公开发布以支持6G专用模型的训练与微调。利用6G-Bench，我们评估了22个基础模型，涵盖稠密架构与专家混合架构、短上下文与长上下文设计（最高达100万词元）以及开源权重与专有系统。所有模型的确定性单次命中准确率（pass@1）分布在0.22至0.82的广泛区间，凸显了语义推理能力的显著差异。领先模型在意图与策略推理任务上达到0.87–0.89的准确率，而对推理密集型任务的选择性鲁棒性分析显示pass@5值分布在0.20至0.91之间。为支持开放科学与可复现性，我们已在GitHub发布6G-Bench数据集：https://github.com/maferrag/6G-Bench