For domains in which a recommender provides repeated content suggestions, agent preferences may evolve over time as a function of prior recommendations, and algorithms must take this into account for long-run optimization. Recently, Agarwal and Brown (2022) introduced a model for studying recommendations when agents' preferences are adaptive, and gave a series of results for the case when agent preferences depend {\it uniformly} on their history of past selections. Here, the recommender shows a $k$-item menu (out of $n$) to the agent at each round, who selects one of the $k$ items via their history-dependent {\it preference model}, yielding a per-item adversarial reward for the recommender. We expand this setting to {\it non-uniform} preferences, and give a series of results for {\it $\gamma$-discounted} histories. For this problem, the feasible regret benchmarks can depend drastically on varying conditions. In the ``large $\gamma$'' regime, we show that the previously considered benchmark, the ``EIRD set'', is attainable for any {\it smooth} model, relaxing the ``local learnability'' requirement from the uniform memory case. We introduce ``pseudo-increasing'' preference models, for which we give an algorithm which can compete against any item distribution with small uniform noise (the ``smoothed simplex''). We show NP-hardness results for larger regret benchmarks in each case. We give another algorithm for pseudo-increasing models (under a restriction on the adversarial nature of the reward functions), which works for any $\gamma$ and is faster when $\gamma$ is sufficiently small, and we show a super-polynomial regret lower bound with respect to EIRD for general models in the ``small $\gamma$'' regime. We conclude with a pair of algorithms for the memoryless case.

翻译：在推荐系统提供重复内容建议的领域中，智能体的偏好可能随时间推移，并依据既往推荐内容而演变，因此算法必须考虑这一因素以实现长期优化。近期，Agarwal与Brown（2022）提出了一种用于研究智能体偏好具有适应性时的推荐模型，并针对智能体偏好均匀依赖于其历史选择记录的情形给出了一系列结果。在该模型中，推荐者在每一轮向智能体展示一个包含k个选项的菜单（从n个选项中选取），智能体根据其历史依赖的偏好模型从k个选项中做出选择，并为推荐者带来每个选项的对抗性奖励。我们将这一设定扩展至非均匀偏好，并针对γ折扣历史记录给出了一系列结果。对于该问题，可行的遗憾基准在不同条件下可能存在显著差异。在“大γ”情形中，我们证明先前考虑的基准——“EIRD集合”——对于任何平滑模型都是可达的，从而放宽了均匀记忆情形中“局部可学习性”的要求。我们引入了“伪递增”偏好模型，并为此类模型提供了一种算法，该算法能够与任何带有小均匀噪声的选项分布（即“平滑单纯形”）竞争。我们在每种情形中证明了针对更大遗憾基准的NP困难性。我们提出了另一种针对伪递增模型（在奖励函数对抗性性质的限制下）的算法，该算法适用于任意γ，且在γ足够小时运行更快。此外，我们证明在“小γ”情形中，对于一般模型，针对EIRD的平均遗憾存在超多项式下界。最后，我们给出了一对适用于无记忆情形的算法。