In this paper, we generalize the concept of Voronoi regions, define agent utility as the integral of a utility density over the corresponding Voronoi region, derive gradients of the utility, and illustrate the approach in a two-team example from soccer. The generalization of Voronoi regions is in the form of so-called Cost-Induced Voronoi (CIV) regions, where the agent state space may differ from the space being partitioned. One example of such regions is when the cost is given by the optimal solution of an LQR control problem. Then the agent states include position as well as velocity, while the partitioned space only includes positions. The agent utility is defined by integrating some utility density over the CIV region of the agent. This utility density might be the probability density of some beneficial event, such as receiving a pass in soccer. The utility is then the overall probability of receiving a pass and the gradient represents a way to improve that probability. We show how this utility gradient can be computed using the Reynolds Transport Theorem from fluid mechanics, and that this approach achieves similar accuracy while reducing computation time by about an order of magnitude compared to a baseline finite-difference approximation.

翻译：本文推广了Voronoi区域的概念，将智能体效用定义为对应Voronoi区域上效用密度的积分，推导了效用的梯度，并通过足球双队实例说明了该方法。Voronoi区域的推广形式为所谓的“成本诱导Voronoi”（CIV）区域，其中智能体状态空间可与被划分的空间不同。此类区域的典型示例为：成本由LQR控制问题的最优解给出，此时智能体状态包含位置和速度，而被划分空间仅包含位置。智能体效用定义为对其CIV区域上的某种效用密度进行积分得到的值。该效用密度可以是某种有利事件（如足球中接传球）的概率密度。此时，效用即为接传球的总体概率，而梯度则代表提升该概率的途径。我们展示了如何利用流体力学中的雷诺输运定理计算该效用梯度，并证明该方法在保持相似精度的同时，计算时间相比基准有限差分近似减少约一个数量级。