Large Language Models (LLMs) have emerged as powerful tools for automating complex reasoning and decision-making tasks. In telecommunications, they hold the potential to transform network optimization, automate troubleshooting, enhance customer support, and ensure regulatory compliance. However, their deployment in telecom is hindered by domain-specific challenges that demand specialized adaptation. To overcome these challenges and to accelerate the adaptation of LLMs for telecom, we propose MM-Telco, a comprehensive suite of multimodal benchmarks and models tailored for the telecom domain. The benchmark introduces various tasks (both text based and image based) that address various practical real-life use cases such as network operations, network management, improving documentation quality, and retrieval of relevant text and images. Further, we perform baseline experiments with various LLMs and VLMs. The models fine-tuned on our dataset exhibit a significant boost in performance. Our experiments also help analyze the weak areas in the working of current state-of-art multimodal LLMs, thus guiding towards further development and research.

翻译：大型语言模型（LLMs）已成为自动化复杂推理与决策任务的强大工具。在电信领域，它们具有变革网络优化、自动化故障排除、增强客户支持以及保障法规合规性的潜力。然而，其在电信领域的部署受到领域特定挑战的阻碍，需进行专门适配。为克服这些挑战并加速LLMs在电信领域的适配，我们提出MM-Telco——一套专为电信领域定制的综合多模态基准与模型套件。该基准引入涵盖文本与图像的多样化任务，涉及实际应用场景，如网络运维、网络管理、文档质量提升以及相关文本与图像的检索。此外，我们使用各类LLMs与视觉语言模型（VLMs）进行了基线实验。经我们数据集微调的模型表现出显著的性能提升。实验还揭示了当前最先进多模态LLMs工作中的薄弱环节，从而为后续开发与研究提供指引。