Many approaches for optimizing decision making systems rely on gradient based methods requiring informative feedback from the environment. However, in the case where such feedback is sparse or uninformative, such approaches may result in poor performance. Derivative-free approaches such as Bayesian Optimization mitigate the dependency on the quality of gradient feedback, but are known to scale poorly in the high-dimension setting of complex decision making systems. This problem is exacerbated if the system requires interactions between several actors cooperating to accomplish a shared goal. To address the dimensionality challenge, we propose a compact multi-layered architecture modeling the dynamics of actor interactions through the concept of role. We introduce Hessian-aware Bayesian Optimization to efficiently optimize the multi-layered architecture parameterized by a large number of parameters, and give the first improved regret bound in additive high-dimensional Bayesian Optimization since Mutny & Krause (2018). Our approach shows strong empirical results under malformed or sparse reward.

翻译：许多优化决策系统的方法依赖于基于梯度的技术，这些技术需要从环境中获取含信息量的反馈。然而，当此类反馈稀疏或缺乏信息量时，这些方法可能导致性能不佳。无导数方法（如贝叶斯优化）减轻了对梯度反馈质量的依赖，但在复杂决策系统的高维设置中已知扩展性较差。如果系统需要多个参与者之间进行交互以共同完成共享目标，这一问题会更加严重。为解决维度挑战，我们提出了一种紧凑的多层架构，通过角色概念对参与者交互动态进行建模。我们引入了Hessian感知贝叶斯优化来高效优化由大量参数参数化的多层架构，并给出了自Mutny和Krause（2018）以来加性高维贝叶斯优化中首次改进的遗憾界。我们的方法在畸形或稀疏奖励设置下显示出强实证结果。