The latest advancements in machine learning and deep learning have brought forth the concept of semantic similarity, which has proven immensely beneficial in multiple applications and has largely replaced keyword search. However, evaluating semantic similarity and conducting searches for a specific query across various documents continue to be a complicated task. This complexity is due to the multifaceted nature of the task, the lack of standard benchmarks, whereas these challenges are further amplified for Arabic language. This paper endeavors to establish a straightforward yet potent benchmark for semantic search in Arabic. Moreover, to precisely evaluate the effectiveness of these metrics and the dataset, we conduct our assessment of semantic search within the framework of retrieval augmented generation (RAG).

翻译：机器学习和深度学习的最新进展催生了语义相似性概念，该概念已被证明在众多应用中极具价值，并在很大程度上取代了关键词检索。然而，评估语义相似性以及在多篇文档中针对特定查询进行检索，仍然是一项复杂的任务。这种复杂性源于任务本身的多面性、标准基准的缺失，而对于阿拉伯语而言，这些挑战则被进一步放大。本文致力于为阿拉伯语语义检索建立一个简洁而有效的基准。此外，为了精确评估这些指标及数据集的有效性，我们在检索增强生成（RAG）框架内对语义检索进行了评估。