The field of Sequential Decision Making (SDM) provides tools for solving Sequential Decision Processes (SDPs), where an agent must make a series of decisions in order to complete a task or achieve a goal. Historically, two competing SDM paradigms have view for supremacy. Automated Planning (AP) proposes to solve SDPs by performing a reasoning process over a model of the world, often represented symbolically. Conversely, Reinforcement Learning (RL) proposes to learn the solution of the SDP from data, without a world model, and represent the learned knowledge subsymbolically. In the spirit of reconciliation, we provide a review of symbolic, subsymbolic and hybrid methods for SDM. We cover both methods for solving SDPs (e.g., AP, RL and techniques that learn to plan) and for learning aspects of their structure (e.g., world models, state invariants and landmarks). To the best of our knowledge, no other review in the field provides the same scope. As an additional contribution, we discuss what properties an ideal method for SDM should exhibit and argue that neurosymbolic AI is the current approach which most closely resembles this ideal method. Finally, we outline several proposals to advance the field of SDM via the integration of symbolic and subsymbolic AI.

翻译：序贯决策领域为解决序贯决策过程提供了工具，其中智能体需连续做出决策以完成任务或达成目标。历史上，两大竞争性序贯决策范式曾争夺主导地位。自动规划主张通过对世界模型进行推理过程来求解序贯决策问题，该模型通常以符号形式表征。相反，强化学习则主张无需世界模型，直接从数据中学习序贯决策问题的解，并以亚符号形式表征习得知识。本着调和精神，本文系统综述了面向序贯决策的符号性、亚符号性与混合方法，涵盖求解序贯决策过程的方法（如自动规划、强化学习及学习规划技术）与学习其结构要素的方法（如世界模型、状态不变性与地标）。据我们所知，目前尚无其他综述具备相同覆盖范围。作为额外贡献，我们探讨了理想序贯决策方法应具备的特性，并论证神经符号人工智能是当前最接近该理想方法的研究方向。最后，我们提出若干通过融合符号性与亚符号性人工智能来推进序贯决策领域的建议。