Novel imaging and neurostimulation techniques open doors for advancements in closed-loop control of activity in biological neural networks. This would allow for applications in the investigation of activity propagation, and for diagnosis and treatment of pathological behaviour. Due to the partially observable characteristics of activity propagation, through networks in which edges can not be observed, and the dynamic nature of neuronal systems, there is a need for adaptive, generalisable control. In this paper, we introduce an environment that procedurally generates neuronal networks with different topologies to investigate this generalisation problem. Additionally, an existing transformer-based architecture is adjusted to evaluate the generalisation performance of a deep RL agent in the presented partially observable environment. The agent demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.

翻译：新型成像与神经调控技术为生物神经网络活动的闭环控制开辟了前进道路。这将有助于活动传播机制的研究，并为病理行为的诊断与治疗提供应用可能。由于活动传播具有部分可观测特性（网络中部分连接无法被观测），且神经元系统具有动态本质，因此需要具备适应性、可泛化的控制方法。本文引入了一个按程序生成不同拓扑结构神经网络的仿真环境，以探究这一泛化问题。此外，我们调整了一种现有的基于Transformer的架构，用于评估深度强化学习智能体在所构建的部分可观测环境中的泛化性能。实验表明，该智能体能够将控制策略从有限数量的训练网络泛化至未见过的测试网络。