The term "Code Mixed" refers to the use of more than one language in the same text. This phenomenon is predominantly observed on social media platforms, with an increasing amount of adaptation as time goes on. It is critical to detect foreign elements in a language and process them correctly, as a considerable number of individuals are using code-mixed languages that could not be comprehended by understanding one of those languages. In this work, we focus on low-resource Hindi-English code-mixed language and enhancing the performance of different code-mixed natural language processing tasks such as sentiment analysis, emotion recognition, and hate speech identification. We perform a comparative analysis of different Transformer-based language Models pre-trained using unsupervised approaches. We have included the code-mixed models like HingBERT, HingRoBERTa, HingRoBERTa-Mixed, mBERT, and non-code-mixed models like AlBERT, BERT, and RoBERTa for comparative analysis of code-mixed Hindi-English downstream tasks. We report state-of-the-art results on respective datasets using HingBERT-based models which are specifically pre-trained on real code-mixed text. Our HingBERT-based models provide significant improvements thus highlighting the poor performance of vanilla BERT models on code-mixed text.

翻译：“代码混合”指在同一文本中使用两种或以上语言的现象。这种现象在社交媒体平台上尤为常见，且随着时间推移其适应程度日益增加。由于大量用户使用无法通过单一语言理解的代码混合语言，检测语言中的外来元素并正确处理它们至关重要。本研究聚焦于低资源印地语-英语代码混合语言，旨在提升不同代码混合自然语言处理任务（如情感分析、情绪识别和仇恨言论检测）的性能。我们对基于无监督预训练方法的不同Transformer语言模型进行了比较分析。研究中纳入了HingBERT、HingRoBERTa、HingRoBERTa-Mixed、mBERT等代码混合模型，以及AlBERT、BERT、RoBERTa等非代码混合模型，以进行印地语-英语代码混合下游任务的对比分析。我们报告了基于专门在真实代码混合文本上预训练的HingBERT模型在相应数据集上的最新成果。基于HingBERT的模型取得了显著改进，凸显了原始BERT模型在代码混合文本上的性能不足。