Reinforcement Learning (RL) enables autonomous agents to learn policies from experience, but realistic problems often involve enormous state spaces, making learning and generalisation challenging. Abstraction and approximation are therefore essential. Relational Reinforcement Learning (RRL) offers a way to reason about objects and their relations, and the CARCASS framework by Martijn van Otterlo demonstrates how logical representations can model Markov Decision Processes (MDPs) in first-order domains. Originally implemented in Prolog, CARCASS leverages domain knowledge to create powerful abstractions. We explore Answer-Set Programming (ASP), which is a rich and, contrary to Prolog, fully declarative modelling language, to realise CARCASS abstractions. We evaluate our ASP-based implementation in case studies of two domains, viz. Blocks World and Minigrid. Our results indicate that CARCASS with ASP provides a promising approach to constructing abstractions for RL, especially when domain knowledge is available.

翻译：强化学习使自主智能体能够从经验中学习策略，但实际问题往往涉及巨大状态空间，导致学习与泛化面临挑战。因此抽象与近似方法至关重要。关系强化学习提供了对对象及其关系进行推理的途径，而Martijn van Otterlo提出的CARCASS框架展示了如何用逻辑表示对一阶域中的马尔可夫决策过程进行建模。该框架最初基于Prolog实现，通过利用领域知识构建强大的抽象方法。本文探索采用回答集编程（一种比Prolog更丰富的全声明式建模语言）来实现CARCASS抽象方法。我们在两个领域（积木世界与Minigrid）的案例研究中评估了基于ASP的实现方案。结果表明，基于ASP的CARCASS方法为构建强化学习抽象提供了有效途径，尤其在具备领域知识时表现突出。