Plagiarism involves using another person's work or concepts without proper attribution, presenting them as original creations. With the growing amount of data communicated in regional languages such as Marathi -- one of India's regional languages -- it is crucial to design robust plagiarism detection systems tailored for low-resource languages. Language models like Bidirectional Encoder Representations from Transformers (BERT) have demonstrated exceptional capability in text representation and feature extraction, making them essential tools for semantic analysis and plagiarism detection. However, the application of BERT for low-resource languages remains under-explored, particularly in the context of plagiarism detection. This paper presents a method to enhance the accuracy of plagiarism detection for Marathi texts using BERT sentence embeddings in conjunction with Term Frequency-Inverse Document Frequency (TF-IDF) feature representation. This approach effectively captures statistical, semantic, and syntactic aspects of text features through a weighted voting ensemble of machine learning models.
翻译:剽窃指未经适当署名而使用他人作品或概念,并将其呈现为原创成果。随着马拉地语(印度地区性语言之一)等区域语言承载的数据量日益增长,亟需针对低资源语言设计鲁棒的剽窃检测系统。以Transformer双向编码器表征(BERT)为代表的语言模型在文本表征与特征提取方面展现出卓越能力,已成为语义分析与剽窃检测的关键工具。然而,BERT在低资源语言中的应用仍待深入探索,尤其在剽窃检测领域。本文提出一种结合BERT句子嵌入与词频-逆文档频率(TF-IDF)特征表征的方法,以提升马拉地语文本剽窃检测的准确率。该方法通过机器学习模型的加权投票集成,有效捕捉文本特征的统计、语义及句法层面信息。