Behavior trees represent a modular way to create an overall controller from a set of sub-controllers solving different sub-problems. These sub-controllers can be created in different ways, such as classical model based control or reinforcement learning (RL). If each sub-controller satisfies the preconditions of the next sub-controller, the overall controller will achieve the overall goal. However, even if all sub-controllers are locally optimal in achieving the preconditions of the next, with respect to some performance metric such as completion time, the overall controller might be far from optimal with respect to the same performance metric. In this paper we show how the performance of the overall controller can be improved if we use approximations of value functions to inform the design of a sub-controller of the needs of the next one. We also show how, under certain assumptions, this leads to a globally optimal controller when the process is executed on all sub-controllers. Finally, this result also holds when some of the sub-controllers are already given, i.e., if we are constrained to use some existing sub-controllers the overall controller will be globally optimal given this constraint.

翻译：行为树提供了一种模块化方式，通过从一组解决不同子问题的子控制器构建整体控制器。这些子控制器可通过不同方式创建，例如经典基于模型的控制或强化学习。若每个子控制器满足下一个子控制器的前提条件，则整体控制器将实现总体目标。然而，即使所有子控制器在达到下一子控制器前提条件方面（就完成时间等性能指标而言）均为局部最优，整体控制器在该性能指标上仍可能远非最优。本文展示了如何通过使用价值函数近似来告知子控制器设计其后续需求，从而提升整体控制器的性能。我们同时证明，在特定假设条件下，当该过程应用于所有子控制器时，将得到全局最优控制器。最后，即便部分子控制器已被预先给定（即我们受限于使用某些现有子控制器），该结论仍然成立——在此约束下整体控制器将实现全局最优。