Large Language Models (LLMs) are now integral to numerous industries, increasingly serving as the core reasoning engine for autonomous agents that perform complex tasks through tool-use. While the development of Arabic-native LLMs is accelerating, the benchmarks for evaluating their capabilities lag behind, with most existing frameworks focusing on English. A critical and overlooked area is tool-calling, where the performance of models prompted in non-English languages like Arabic is poorly understood, especially since these models are often pretrained on predominantly English data. This paper addresses this critical gap by introducing the first dedicated benchmark for evaluating the tool-calling and agentic capabilities of LLMs in the Arabic language. Our work provides a standardized framework to measure the functional accuracy and robustness of models in Arabic agentic workflows. Our findings reveal a huge performance gap: when users interact in Arabic, tool-calling accuracy drops by an average of 5-10\%, regardless of whether the tool descriptions themselves are in Arabic or English. By shedding light on these critical challenges, this benchmark aims to foster the development of more reliable and linguistically equitable AI agents for Arabic-speaking users.

翻译：大型语言模型（LLM）现已成为众多行业不可或缺的组成部分，日益成为自主智能体执行复杂任务的核心推理引擎，这些智能体通过使用工具来运作。尽管阿拉伯语原生LLM的开发正在加速，但评估其能力的基准却相对滞后，现有框架大多专注于英语。一个关键且被忽视的领域是工具调用，对于使用非英语语言（如阿拉伯语）进行提示的模型，其性能表现尚不明确，尤其因为这些模型通常是在以英语为主的数据上进行预训练的。本文通过引入首个专门用于评估LLM在阿拉伯语中工具调用与智能体能力的基准，填补了这一关键空白。我们的工作提供了一个标准化框架，用于衡量模型在阿拉伯语智能体工作流程中的功能准确性和鲁棒性。我们的研究结果揭示了一个巨大的性能差距：当用户使用阿拉伯语交互时，工具调用的准确率平均下降5-10%，无论工具描述本身是阿拉伯语还是英语。通过揭示这些关键挑战，本基准旨在推动为阿拉伯语用户开发更可靠、语言更公平的人工智能智能体。