With the help of Deep Neural Networks, Deep Reinforcement Learning (DRL) has achieved great success on many complex tasks during the past few years. Spiking Neural Networks (SNNs) have been used for the implementation of Deep Neural Networks with superb energy efficiency on dedicated neuromorphic hardware, and recent years have witnessed increasing attention on combining SNNs with Reinforcement Learning, whereas most approaches still work with huge energy consumption and high latency. This work proposes the Adaptive Coding Spiking Framework (ACSF) for SNN-based DRL and achieves low latency and great energy efficiency at the same time. Inspired by classical conditioning in biology, we simulate receptors, central interneurons, and effectors with spike encoders, SNNs, and spike decoders, respectively. We use our proposed ACSF to estimate the value function in reinforcement learning and conduct extensive experiments to verify the effectiveness of our proposed framework.

翻译：在深神经网络的帮助下,深强化学习(DRL)在过去几年中在许多复杂任务上取得了巨大成功,Spiking神经网络(SNNS)被用于实施在专用神经形态硬件上具有超能能效的深神经网络,近年来,人们日益关注将SNNS与强化学习相结合,而大多数方法仍然使用大量能源消耗和高潜伏。这项工作提议为SNN的DRL建立适应编码 Spiking框架(ACSF),并同时实现低持久性和高能效。我们受到生物学古典调节的启发,我们模拟受感应器、中中中中中中子以及分别带有峰值聚合器、SNNS和峰值分解器的效应器。我们利用我们提议的ACF来估计在加强学习和进行广泛实验方面的价值功能,以核实我们拟议框架的有效性。