Natural-language user profiles have recently attracted attention not only for improved interpretability, but also for their potential to make recommender systems more steerable. By enabling direct editing, natural-language profiles allow users to explicitly articulate preferences that may be difficult to infer from past behavior. However, it remains unclear whether current natural-language-based recommendation methods can follow such steering commands. While existing steerability evaluations have shown some success for well-recognized item attributes (e.g., movie genres), we argue that these benchmarks fail to capture the richer forms of user control that motivate steerable recommendations. To address this gap, we introduce SteerEval, an evaluation framework designed to measure more nuanced and diverse forms of steerability by using interventions that range from genres to content-warning for movies. We assess the steerability of a family of pretrained natural-language recommenders, examine the potential and limitations of steering on relatively niche topics, and compare how different profile and recommendation interventions impact steering effectiveness. Finally, we offer practical design suggestions informed by our findings and discuss future steps in steerable recommender design.

翻译：自然语言用户画像近来不仅因其可解释性而受到关注，更因其使推荐系统更具可操控性的潜力而备受重视。通过支持直接编辑，自然语言画像允许用户明确表达那些难以从历史行为中推断的偏好。然而，目前基于自然语言的推荐方法能否遵循此类操控指令仍不明确。尽管现有的可操控性评估在广为人知的物品属性（例如电影类型）上已显示出一定成效，但我们认为这些基准未能捕捉到激励可操控推荐的更丰富的用户控制形式。为填补这一空白，我们提出了SteerEval，这是一个评估框架，旨在通过使用从电影类型到内容警告等多种干预手段，来衡量更细致和多样化的可操控性形式。我们评估了一系列预训练的自然语言推荐器的可操控性，考察了在相对小众主题上进行操控的潜力与局限，并比较了不同画像与推荐干预方式对操控效果的影响。最后，我们基于研究发现提出了实用的设计建议，并讨论了可操控推荐系统设计的未来方向。