This paper explores the potential of the transformer models for learning Granger causality in networks with complex nonlinear dynamics at every node, as in neurobiological and biophysical networks. Our study primarily focuses on a proof-of-concept investigation based on simulated neural dynamics, for which the ground-truth causality is known through the underlying connectivity matrix. For transformer models trained to forecast neuronal population dynamics, we show that the cross attention module effectively captures the causal relationship among neurons, with an accuracy equal or superior to that for the most popular Granger causality analysis method. While we acknowledge that real-world neurobiology data will bring further challenges, including dynamic connectivity and unobserved variability, this research offers an encouraging preliminary glimpse into the utility of the transformer model for causal representation learning in neuroscience.

翻译：本文探讨了Transformer模型在具有复杂非线性动力学的网络中学习格兰杰因果关系的潜力，这类网络常见于神经生物学与生物物理系统。我们的研究主要基于模拟神经动力学的概念验证，其中通过底层连接矩阵已知真实的因果关系。针对训练用于预测神经群体动力学的Transformer模型，我们证明了交叉注意力模块能够有效捕捉神经元之间的因果关系，其准确度达到甚至超越最流行的格兰杰因果关系分析方法。尽管我们承认真实神经生物学数据将带来更多挑战，包括动态连接性和未观测到的变异性，但本研究为Transformer模型在神经科学中用于因果表示学习提供了令人鼓舞的初步启示。