We observe a change in the way users access information, that is, the rise of conversational information access (CIA) agents. However, the automatic evaluation of these agents remains an open challenge. Moreover, the training of CIA agents is cumbersome as it mostly relies on conversational corpora, expert knowledge, and reinforcement learning. User simulation has been identified as a promising solution to tackle automatic evaluation and has been previously used in reinforcement learning. In this research, we investigate how user simulation can be leveraged in the context of CIA. We organize the work in three parts. We begin with the identification of requirements for user simulators for training and evaluating CIA agents and compare existing types of simulator regarding these. Then, we plan to combine these different types of simulators into a new hybrid simulator. Finally, we aim to extend simulators to handle more complex information seeking scenarios.

翻译：我们观察到用户获取信息方式的转变，即对话式信息访问代理的兴起。然而，这些代理的自动评估仍然是一个开放性挑战。此外，对话式信息访问代理的训练因主要依赖对话语料库、专家知识和强化学习而较为繁琐。用户模拟已被视为解决自动评估问题的一种有前景的方案，并曾在强化学习中使用过。在本研究中，我们探讨如何在对话式信息访问场景中利用用户模拟。我们将工作分为三部分进行。首先，我们识别用户模拟器在训练和评估对话式信息访问代理方面的需求，并对比现有模拟器类型。其次，我们计划将这些不同类型的模拟器结合为一种全新的混合模拟器。最后，我们旨在扩展模拟器以处理更复杂的信息检索场景。