Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work studies the former. Specifically, the Perception and Decision-making Interleaving Transformer (PDiT) network is proposed, which cascades two Transformers in a very natural way: the perceiving one focuses on \emph{the environmental perception} by processing the observation at the patch level, whereas the deciding one pays attention to \emph{the decision-making} by conditioning on the history of the desired returns, the perceiver's outputs, and the actions. Such a network design is generally applicable to a lot of deep RL settings, e.g., both the online and offline RL algorithms under environments with either image observations, proprioception observations, or hybrid image-language observations. Extensive experiments show that PDiT can not only achieve superior performance than strong baselines in different settings but also extract explainable feature representations. Our code is available at \url{https://github.com/maohangyu/PDiT}.

翻译：设计更优的深度网络与更优的强化学习算法对深度强化学习均至关重要。本文聚焦于前者研究。具体而言，提出感知与决策交错Transformer网络，该网络以自然方式级联两个Transformer：感知型Transformer专注于在补丁层级处理观测信息以完成环境感知，而决策型Transformer则通过条件化于期望回报的历史、感知器的输出以及动作来关注决策制定。该网络设计广泛适用于多种深度强化学习场景，例如在图像观测、本体感受观测或图像-语言混合观测环境下的在线与离线强化学习算法。大量实验表明，PDiT不仅能在不同设置中取得优于强基线方法的性能，还能提取可解释的特征表示。我们的代码开源在\url{https://github.com/maohangyu/PDiT}。