Large Language Models (LLMs) can benefit from useful experiences to improve their performance on specific tasks. However, finding helpful experiences for different LLMs is not obvious, since it is unclear what experiences suit specific LLMs. Previous studies intended to automatically find useful experiences using LLMs, while it is difficult to ensure the effectiveness of the obtained experience. In this paper, we propose Stochastic Experience Optimization (SEO), an iterative approach that finds optimized model-specific experience without modifying model parameters through experience update in natural language. In SEO, we propose a stochastic validation method to ensure the update direction of experience, avoiding unavailing updates. Experimental results on three tasks for three LLMs demonstrate that experiences optimized by SEO can achieve consistently improved performance. Further analysis indicates that SEO-optimized experience can generalize to out-of-distribution data, boosting the performance of LLMs on similar tasks.

翻译：大语言模型（LLMs）能够从有益的经验中学习，以提升其在特定任务上的性能。然而，为不同的大语言模型寻找有帮助的经验并非易事，因为尚不清楚何种经验适合特定模型。先前的研究尝试利用大语言模型自动寻找有用经验，但难以确保所获经验的有效性。本文提出随机经验优化（SEO），这是一种迭代方法，通过自然语言中的经验更新来寻找优化的、模型特定的经验，而无需修改模型参数。在SEO中，我们提出了一种随机验证方法，以确保经验的更新方向，避免无效更新。在三个大语言模型上针对三项任务的实验结果表明，经SEO优化的经验能够持续提升模型性能。进一步分析表明，SEO优化的经验能够泛化至分布外数据，从而提升大语言模型在相似任务上的表现。