In this paper, we propose a methodology to align a medium-sized GPT model, originally trained in English for an open domain, to a small closed domain in Spanish. The application for which the model is finely tuned is the question answering task. To achieve this we also needed to train and implement another neural network (which we called the reward model) that could score and determine whether an answer is appropriate for a given question. This component served to improve the decoding and generation of the answers of the system. Numerical metrics such as BLEU and perplexity were used to evaluate the model, and human judgment was also used to compare the decoding technique with others. Finally, the results favored the proposed method, and it was determined that it is feasible to use a reward model to align the generation of responses.

翻译：本文提出了一种方法论，旨在将原本在英语开放领域预训练的中等规模GPT模型，对齐至西班牙语小型封闭领域。该模型微调的应用场景为问答任务。为实现这一目标，我们还需训练并实现另一神经网络（称为奖励模型），该模型能够对给定问题评估答案的适用性。这一组件用于改进系统答案的解码与生成过程。我们采用BLEU和困惑度等数值指标评估模型性能，并引入人工判断以比较不同解码技术。最终结果支持了所提方法，证实了利用奖励模型对齐生成响应的可行性。