Natural Language Inference (NLI) is a cornerstone of Natural Language Processing (NLP), providing insights into the entailment relationships between text pairings. It is a critical component of Natural Language Understanding (NLU), demonstrating the ability to extract information from spoken or written interactions. NLI is mainly concerned with determining the entailment relationship between two statements, known as the premise and hypothesis. When the premise logically implies the hypothesis, the pair is labeled "entailment". If the hypothesis contradicts the premise, the pair receives the "contradiction" label. When there is insufficient evidence to establish a connection, the pair is described as "neutral". Despite the success of Large Language Models (LLMs) in various tasks, their effectiveness in NLI remains constrained by issues like low-resource domain accuracy, model overconfidence, and difficulty in capturing human judgment disagreements. This study addresses the underexplored area of evaluating LLMs in low-resourced languages such as Bengali. Through a comprehensive evaluation, we assess the performance of prominent LLMs and state-of-the-art (SOTA) models in Bengali NLP tasks, focusing on natural language inference. Utilizing the XNLI dataset, we conduct zero-shot and few-shot evaluations, comparing LLMs like GPT-3.5 Turbo and Gemini 1.5 Pro with models such as BanglaBERT, Bangla BERT Base, DistilBERT, mBERT, and sahajBERT. Our findings reveal that while LLMs can achieve comparable or superior performance to fine-tuned SOTA models in few-shot scenarios, further research is necessary to enhance our understanding of LLMs in languages with modest resources like Bengali. This study underscores the importance of continued efforts in exploring LLM capabilities across diverse linguistic contexts.

翻译：自然语言推理（NLI）是自然语言处理（NLP）的基石，揭示了文本对之间的蕴涵关系。作为自然语言理解（NLU）的关键组成部分，它展示了从口语或书面交互中提取信息的能力。NLI主要关注确定两个语句（称为前提和假设）之间的蕴涵关系。当前提在逻辑上蕴含假设时，该对语句被标记为“蕴涵”；若假设与前提矛盾，则标记为“矛盾”；当缺乏足够证据建立联系时，则标记为“中性”。尽管大型语言模型（LLM）在各种任务中取得了成功，但其在NLI中的有效性仍受限于低资源领域准确性、模型过度自信以及难以捕捉人类判断分歧等问题。本研究针对尚未充分探索的领域——评估LLM在孟加拉语等低资源语言中的表现。通过全面评估，我们评估了主流LLM和当前最优（SOTA）模型在孟加拉语NLP任务中的性能，重点关注自然语言推理。利用XNLI数据集，我们进行了零样本和少样本评估，比较了GPT-3.5 Turbo和Gemini 1.5 Pro等LLM与BanglaBERT、Bangla BERT Base、DistilBERT、mBERT和sahajBERT等模型。我们的研究发现表明，尽管LLM在少样本场景下能达到与微调SOTA模型相当或更优的性能，但仍需进一步研究以加深对LLM在孟加拉语等资源有限语言中的理解。本研究强调了持续探索LLM在多语言语境中能力的重要性。