The advent of personalized content generation by LLMs presents a novel challenge: how to efficiently adapt text to meet individual preferences without the unsustainable demand of creating a unique model for each user. This study introduces an innovative online method that employs neural bandit algorithms to dynamically optimize soft instruction embeddings based on user feedback, enhancing the personalization of open-ended text generation by white-box LLMs. Through rigorous experimentation on various tasks, we demonstrate significant performance improvements over baseline strategies. NeuralTS, in particular, leads to substantial enhancements in personalized news headline generation, achieving up to a 62.9% improvement in terms of best ROUGE scores and up to 2.76% increase in LLM-agent evaluation against the baseline.

翻译：个性化内容生成技术的兴起为大语言模型带来了新挑战：如何在避免为每位用户单独训练模型的不可持续需求下，高效调整文本以满足个体偏好。本研究提出一种创新的在线方法，采用神经赌博机算法，根据用户反馈动态优化软指令嵌入，从而增强白盒大语言模型在开放式文本生成中的个性化能力。通过多任务下的严格实验，我们验证了该方法相较基线策略的显著性能提升。其中，神经汤普森采样算法尤为突出：在个性化新闻标题生成任务中，最佳ROUGE评分最高提升62.9%，在大语言模型智能体评估中较基线提高2.76%。