In this paper, we propose a methodology to align a medium-sized GPT model, originally trained in English for an open domain, to a small closed domain in Spanish. The application for which the model is finely tuned is the question answering task. To achieve this we also needed to train and implement another neural network (which we called the reward model) that could score and determine whether an answer is appropriate for a given question. This component served to improve the decoding and generation of the answers of the system. Numerical metrics such as BLEU and perplexity were used to evaluate the model, and human judgment was also used to compare the decoding technique with others. Finally, the results favored the proposed method, and it was determined that it is feasible to use a reward model to align the generation of responses.

翻译：在本文中，我们提出了一种方法，将最初在英语开放领域训练的中等规模GPT模型，对齐至西班牙语的小型封闭领域。该模型的微调任务为问答任务。为实现这一目标，我们还需训练并实现另一个神经网络（称为奖励模型），该模型能够评分并判断给定问题的答案是否恰当。该组件用于改进系统解码与答案生成过程。我们使用BLEU和困惑度等数值指标评估模型，同时结合人类判断，将所提解码技术与其他技术进行对比。最终，实验结果支持了所提方法，并证实使用奖励模型来对齐响应生成是可行的。