Many approaches for optimizing decision making systems rely on gradient based methods requiring informative feedback from the environment. However, in the case where such feedback is sparse or uninformative, such approaches may result in poor performance. Derivative-free approaches such as Bayesian Optimization mitigate the dependency on the quality of gradient feedback, but are known to scale poorly in the high-dimension setting of complex decision making systems. This problem is exacerbated if the system requires interactions between several actors cooperating to accomplish a shared goal. To address the dimensionality challenge, we propose a compact multi-layered architecture modeling the dynamics of actor interactions through the concept of role. Additionally, we introduce Hessian-aware Bayesian Optimization to efficiently optimize the multi-layered architecture parameterized by a large number of parameters. Experimental results demonstrate that our method (HA-GP-UCB) works effectively on several benchmarks under resource constraints and malformed feedback settings.

翻译：许多优化决策系统的方法依赖于基于梯度的方式，需要从环境中获取信息丰富的反馈。然而，当此类反馈稀疏或信息量不足时，此类方法可能导致性能不佳。无导数方法（如贝叶斯优化）减轻了对梯度反馈质量的依赖，但在复杂决策系统的高维场景中扩展性较差。若系统需要多个代理协作完成共同目标，这一问题会进一步加剧。为应对维度挑战，我们提出了一种紧凑的多层架构，通过“角色”概念建模代理交互的动态特性。此外，我们引入了海森感知贝叶斯优化，以高效优化具有大量参数的多层架构。实验结果表明，我们的方法（HA-GP-UCB）在资源受限和反馈异常设置下，能在多个基准测试中有效运行。