Adaptations of Transformer models, such as BERT4Rec and SASRec, achieve state-of-the-art performance in the sequential recommendation task according to accuracy-based metrics, such as NDCG. These models treat items as tokens and then utilise a score-and-rank approach (Top-K strategy), where the model first computes item scores and then ranks them according to this score. While this approach works well for accuracy-based metrics, it is hard to use it for optimising more complex beyond-accuracy metrics such as diversity. Recently, the GPTRec model, which uses a different Next-K strategy, has been proposed as an alternative to the Top-K models. In contrast with traditional Top-K recommendations, Next-K generates recommendations item-by-item and, therefore, can account for complex item-to-item interdependencies important for the beyond-accuracy measures. However, the original GPTRec paper focused only on accuracy in experiments and needed to address how to optimise the model for complex beyond-accuracy metrics. Indeed, training GPTRec for beyond-accuracy goals is challenging because the interaction training data available for training recommender systems typically needs to be aligned with beyond-accuracy recommendation goals. To solve the misalignment problem, we train GPTRec using a 2-stage approach: in the first stage, we use a teacher-student approach to train GPTRec, mimicking the behaviour of traditional Top-K models; in the second stage, we use Reinforcement Learning to align the model for beyond-accuracy goals. In particular, we experiment with increasing recommendation diversity and reducing popularity bias. Our experiments on two datasets show that in 3 out of 4 cases, GPTRec's Next-K generation approach offers a better tradeoff between accuracy and secondary metrics than classic greedy re-ranking techniques.

翻译：Transformer模型的变体（如BERT4Rec和SASRec）在序列推荐任务中基于准确度指标（如NDCG）取得了最先进的性能。这些模型将项目视为词元，并采用评分排序方法（Top-K策略）：模型先计算项目分数，再根据分数进行排序。尽管该方法在准确度指标上表现良好，但难以用于优化更复杂的超准确率指标（如多样性）。近期，采用不同Next-K策略的GPTRec模型被提出作为Top-K模型的替代方案。与传统的Top-K推荐不同，Next-K逐项生成推荐结果，因此能够考虑对超准确率指标至关重要的复杂项目间依赖关系。然而，原始GPTRec论文仅在实验中关注准确度，未涉及如何针对超准确率指标优化模型。实际上，将GPTRec训练至超准确率目标极具挑战性，因为训练推荐系统可用的交互数据通常与超准确率推荐目标不一致。为解决这种不一致问题，我们采用两阶段方法训练GPTRec：第一阶段使用师生方法训练GPTRec，模仿传统Top-K模型的行为；第二阶段使用强化学习使模型对齐超准确率目标。具体实验中，我们尝试提升推荐多样性和减少流行度偏差。在两个数据集上的实验表明，在4种场景中有3种情况下，GPTRec的Next-K生成方法相比经典贪心重排序技术，能在准确度与次要指标之间实现更优的权衡。