Reward allocation, also known as the credit assignment problem, has been an important topic in economics, engineering, and machine learning. An important concept in reward allocation is the core, which is the set of stable allocations where no agent has the motivation to deviate from the grand coalition. In previous works, computing the core requires either knowledge of the reward function in deterministic games or the reward distribution in stochastic games. However, this is unrealistic, as the reward function or distribution is often only partially known and may be subject to uncertainty. In this paper, we consider the core learning problem in stochastic cooperative games, where the reward distribution is unknown. Our goal is to learn the expected core, that is, the set of allocations that are stable in expectation, given an oracle that returns a stochastic reward for an enquired coalition each round. Within the class of strictly convex games, we present an algorithm named \texttt{Common-Points-Picking} that returns a point in the expected core given a polynomial number of samples, with high probability. To analyse the algorithm, we develop a new extension of the separation hyperplane theorem for multiple convex sets.
翻译:奖励分配,亦称信用分配问题,一直是经济学、工程学和机器学习领域的重要课题。奖励分配中的一个重要概念是核心,即稳定的分配集合,其中没有任何智能体有动机从大联盟中偏离。在以往的研究中,计算核心要么需要确定性博弈中奖励函数的知识,要么需要随机博弈中奖励分布的知识。然而,这并不现实,因为奖励函数或分布通常仅部分已知,且可能具有不确定性。本文考虑随机合作博弈中的核心学习问题,其中奖励分布未知。我们的目标是学习期望核心,即在期望意义下稳定的分配集合,前提是每轮有一个预言机为查询的联盟返回一个随机奖励。在严格凸博弈的类别内,我们提出了一种名为 \texttt{Common-Points-Picking} 的算法,该算法能以高概率在多项式数量的样本下返回期望核心中的一个点。为了分析该算法,我们针对多个凸集发展了一种分离超平面定理的新扩展。