Brazilian Portuguese and European Portuguese are two varieties of the same language and, despite their close similarities, they exhibit several differences. However, there is a significant disproportion in the availability of resources between the two variants, with Brazilian Portuguese having more abundant resources. This inequity can impact the quality of translation services accessible to European Portuguese speakers. To address this issue, we propose the development of a Brazilian Portuguese to European Portuguese translation system, leveraging recent advancements in neural architectures and models. To evaluate the performance of such systems, we manually curated a gold test set comprising 500 sentences across five different topics. Each sentence in the gold test set has two distinct references, facilitating a straightforward evaluation of future translation models. We experimented with various models by fine-tuning existing Large Language Models using parallel data extracted from movie subtitles and TED Talks transcripts in both Brazilian and European Portuguese. Our evaluation involved the use of conventional automatic metrics as well as a human evaluation. In addition, all models were compared against ChatGPT 3.5 Turbo, which currently yields the best results.
翻译:巴西葡萄牙语与欧洲葡萄牙语是同一种语言的两种变体,尽管高度相似,但仍存在若干差异。然而,这两种变体在资源可用性上存在显著不均衡,巴西葡萄牙语资源更为丰富。这种不平衡可能影响欧洲葡萄牙语使用者可获得的翻译服务质量。为解决此问题,我们提出开发一个基于神经架构与模型最新进展的巴西葡萄牙语至欧洲葡萄牙语翻译系统。为评估此类系统的性能,我们人工构建了一个包含五个不同主题共500句的黄金测试集。该测试集中的每个句子均配有两条独立参考译文,便于对后续翻译模型进行直接评估。我们通过使用从电影字幕和TED演讲文稿中提取的巴西与欧洲葡萄牙语平行数据对现有大型语言模型进行微调,对多种模型进行了实验。评估过程采用了传统自动指标及人工评估。此外,所有模型均与当前表现最佳的ChatGPT 3.5 Turbo进行了对比。