Many approaches for optimizing decision making systems rely on gradient based methods requiring informative feedback from the environment. However, in the case where such feedback is sparse or uninformative, such approaches may result in poor performance. Derivative-free approaches such as Bayesian Optimization mitigate the dependency on the quality of gradient feedback, but are known to scale poorly in the high-dimension setting of complex decision making systems. This problem is exacerbated if the system requires interactions between several actors cooperating to accomplish a shared goal. To address the dimensionality challenge, we propose a compact multi-layered architecture modeling the dynamics of actor interactions through the concept of role. Additionally, we introduce Hessian-aware Bayesian Optimization to efficiently optimize the multi-layered architecture parameterized by a large number of parameters. Experimental results demonstrate that our method (HA-GP-UCB) works effectively on several benchmarks under resource constraints and malformed feedback settings.

翻译：许多优化决策系统的方法依赖于基于梯度的算法，这些算法需要从环境中获得信息丰富的反馈。然而，当此类反馈稀疏或缺乏信息时，这类方法可能导致性能不佳。无导数方法（如贝叶斯优化）降低了对梯度反馈质量的依赖性，但在复杂决策系统的高维场景中扩展性较差。若系统需要多个参与者协作完成共同目标，这一问题会进一步加剧。为应对维度挑战，我们提出一种紧凑的多层架构，通过“角色”概念建模参与者交互的动态过程。此外，我们引入基于海森矩阵感知的贝叶斯优化，高效优化由大量参数定义的多层架构。实验结果表明，在资源受限及反馈异常条件下，我们的方法（HA-GP-UCB）在多个基准测试中表现出色。