Most modern recommendation algorithms are data-driven: they generate personalized recommendations by observing users' past behaviors. A common assumption in recommendation is that how a user interacts with a piece of content (e.g., whether they choose to "like" it) is a reflection of the content, but not of the algorithm that generated it. Although this assumption is convenient, it fails to capture user strategization: that users may attempt to shape their future recommendations by adapting their behavior to the recommendation algorithm. In this work, we test for user strategization by conducting a lab experiment and survey. To capture strategization, we adopt a model in which strategic users select their engagement behavior based not only on the content, but also on how their behavior affects downstream recommendations. Using a custom music player that we built, we study how users respond to different information about their recommendation algorithm as well as to different incentives about how their actions affect downstream outcomes. We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes." For example, participants who are told that the algorithm mainly pays attention to "likes" and "dislikes" use those functions 1.9x more than participants told that the algorithm mainly pays attention to dwell time. A close analysis of participant behavior (e.g., in response to our incentive conditions) rules out experimenter demand as the main driver of these trends. Further, in our post-experiment survey, nearly half of participants self-report strategizing "in the wild," with some stating that they ignore content they actually like to avoid over-recommendation of that content in the future. Together, our findings suggest that user strategization is common and that platforms cannot ignore the effect of their algorithms on user behavior.
翻译:大多数现代推荐算法是数据驱动的:它们通过观察用户过去的行为来生成个性化推荐。推荐中的一个常见假设是,用户与内容互动的方式(例如是否选择“喜欢”该内容)反映的是内容本身,而非生成推荐内容的算法。尽管这一假设便于处理,但它未能捕捉到用户的策略化行为:即用户可能通过调整自身行为来适应推荐算法,从而试图塑造未来的推荐内容。在本研究中,我们通过实验室实验和问卷调查来检验用户的策略化行为。为捕捉策略化行为,我们采用了一个模型,其中策略性用户不仅基于内容本身选择其参与行为,还基于其行为如何影响后续推荐。利用我们自建的自定义音乐播放器,我们研究了用户在了解推荐算法不同信息以及其行为如何影响下游结果的不同激励条件下的反应。我们在包括参与者停留时间和“喜欢”使用频率在内的多个结果指标中发现了策略化行为的强有力证据。例如,被告知算法主要关注“喜欢”和“不喜欢”的参与者,其使用这些功能的频率是被告知算法主要关注停留时间的参与者的1.9倍。对参与者行为(例如在激励条件下的反应)的深入分析排除了实验者需求效应作为这些趋势主要驱动因素的可能性。此外,在实验后的调查中,近一半的参与者报告称他们在“真实场景”中也有策略化行为,其中一些参与者表示他们会忽略自己实际喜欢的内容,以避免该内容在未来被过度推荐。总体而言,我们的发现表明,用户的策略化行为普遍存在,平台不能忽视其算法对用户行为的影响。