The term "Code Mixed" refers to the use of more than one language in the same text. This phenomenon is predominantly observed on social media platforms, with an increasing amount of adaptation as time goes on. It is critical to detect foreign elements in a language and process them correctly, as a considerable number of individuals are using code-mixed languages that could not be comprehended by understanding one of those languages. In this work, we focus on low-resource Hindi-English code-mixed language and enhancing the performance of different code-mixed natural language processing tasks such as sentiment analysis, emotion recognition, and hate speech identification. We perform a comparative analysis of different Transformer-based language Models pre-trained using unsupervised approaches. We have included the code-mixed models like HingBERT, HingRoBERTa, HingRoBERTa-Mixed, mBERT, and non-code-mixed models like AlBERT, BERT, and RoBERTa for comparative analysis of code-mixed Hindi-English downstream tasks. We report state-of-the-art results on respective datasets using HingBERT-based models which are specifically pre-trained on real code-mixed text. Our HingBERT-based models provide significant improvements thus highlighting the poor performance of vanilla BERT models on code-mixed text.

翻译：“混合编码”指在同一文本中使用多种语言的现象。这种现象在社交媒体平台上尤为普遍，且随着时间推移使用频率持续增长。由于大量用户使用的混合语言无法通过单一语言理解，因此检测语言中的外来元素并进行正确处理至关重要。本研究聚焦于低资源型印地语-英语混合编码语言，旨在提升情感分析、情绪识别和仇恨言论检测等混合编码自然语言处理任务的性能。我们对基于Transformer架构且采用无监督方法预训练的不同语言模型进行了比较分析。研究纳入HingBERT、HingRoBERTa、HingRoBERTa-Mixed、mBERT等混合编码模型，以及AlBERT、BERT、RoBERTa等非混合编码模型，针对印地语-英语混合编码下游任务开展对比研究。实验表明，基于真实混合编码文本预训练的HingBERT系列模型在各自数据集上达到了当前最优性能。这些模型实现了显著性能提升，凸显了原始BERT模型在处理混合编码文本时的局限性。