Bangla (Bengali) is the fifth most spoken language globally and, yet, the problem of automatic grammar correction in Bangla is still in its nascent stage. This is mostly due to the need for a large corpus of grammatically incorrect sentences, with their corresponding correct counterparts. The present state-of-the-art techniques to curate a corpus for grammatically wrong sentences involve random swapping, insertion and deletion of words. However,these steps may not always generate grammatically wrong sentences in Bangla. In this work, we propose a pragmatic approach to generate grammatically wrong sentences in Bangla. We first categorize the different kinds of errors in Bangla into 5 broad classes and 12 finer classes. We then use these to generate grammatically wrong sentences systematically from a correct sentence. This approach can generate a large number of wrong sentences and can, thus, mitigate the challenge of lacking a large corpus for neural networks. We provide a dataset, Vaiyakarana, consisting of 92,830 grammatically incorrect sentences as well as 18,426 correct sentences. We also collected 619 human-generated sentences from essays written by Bangla native speakers. This helped us to understand errors that are more frequent. We evaluated our corpus against neural models and LLMs and also benchmark it against human evaluators who are native speakers of Bangla. Our analysis shows that native speakers are far more accurate than state-of-the-art models to detect whether the sentence is grammatically correct. Our methodology of generating erroneous sentences can be applied for most other Indian languages as well.
翻译:孟加拉语是全球第五大使用语言,然而,其自动语法纠错问题仍处于起步阶段。这主要是由于缺乏一个包含大量语法错误句子及其对应正确版本的大型语料库。当前用于构建语法错误句子语料库的最先进技术涉及随机交换、插入和删除词语。然而,这些步骤在孟加拉语中并不总能生成语法错误的句子。在本工作中,我们提出了一种实用的方法来生成孟加拉语语法错误句子。我们首先将孟加拉语中的各类错误划分为5个大类和12个细分类别。然后,我们利用这些类别,从正确句子系统地生成语法错误句子。这种方法能够生成大量错误句子,从而缓解神经网络缺乏大型语料库的挑战。我们提供了一个名为Vaiyakarana的数据集,包含92,830个语法错误句子以及18,426个正确句子。我们还从孟加拉语母语者撰写的文章中收集了619个人工生成的句子。这有助于我们理解更常见的错误类型。我们使用神经模型和LLMs评估了我们的语料库,并针对孟加拉语母语者人工评估者进行了基准测试。我们的分析表明,在判断句子语法正确性方面,母语者的准确度远高于最先进的模型。我们生成错误句子的方法也适用于大多数其他印度语言。