We introduce Resonant Sparse Geometry Networks (RSGN), a brain-inspired architecture with self-organizing sparse hierarchical input-dependent connectivity. Unlike Transformer architectures that employ dense attention mechanisms with O(n^2) computational complexity, RSGN embeds computational nodes in learned hyperbolic space where connection strength decays with geodesic distance, achieving dynamic sparsity that adapts to each input. The architecture operates on two distinct timescales: fast differentiable activation propagation optimized through gradient descent, and slow Hebbian-inspired structural learning for connectivity adaptation through local correlation rules. We provide rigorous mathematical analysis demonstrating that RSGN achieves O(n*k) computational complexity, where k << n represents the average active neighborhood size. Experimental evaluation on hierarchical classification and long-range dependency tasks demonstrates that RSGN achieves 96.5% accuracy on long-range dependency tasks while using approximately 15x fewer parameters than standard Transformers. On challenging hierarchical classification with 20 classes, RSGN achieves 23.8% accuracy (compared to 5% random baseline) with only 41,672 parameters, nearly 10x fewer than the Transformer baselines which require 403,348 parameters to achieve 30.1% accuracy. Our ablation studies confirm the contribution of each architectural component, with Hebbian learning providing consistent improvements. These results suggest that brain-inspired principles of sparse, geometrically-organized computation offer a promising direction toward more efficient and biologically plausible neural architectures.

翻译：我们提出了共振稀疏几何网络（RSGN），这是一种受大脑启发的架构，具有自组织稀疏层次化输入依赖连接性。与采用计算复杂度为O(n²)的密集注意力机制的Transformer架构不同，RSGN将计算节点嵌入到学习得到的双曲空间中，其中连接强度随测地距离衰减，实现了适应每个输入的动态稀疏性。该架构在两个不同的时间尺度上运行：通过梯度下降优化的快速可微分激活传播，以及通过局部相关性规则进行连接性适应的、受赫布学习启发的慢速结构学习。我们提供了严格的数学分析，证明RSGN实现了O(n*k)的计算复杂度，其中k << n代表平均活跃邻域大小。在层次分类和长程依赖任务上的实验评估表明，RSGN在长程依赖任务上达到了96.5%的准确率，同时使用的参数数量比标准Transformer大约少15倍。在具有20个类别的挑战性层次分类任务上，RSGN以仅41,672个参数实现了23.8%的准确率（随机基线为5%），这比需要403,348个参数才能达到30.1%准确率的Transformer基线少了近10倍参数。我们的消融研究证实了每个架构组件的贡献，其中赫布学习提供了持续的改进。这些结果表明，受大脑启发的稀疏、几何化组织计算原则为开发更高效且更具生物合理性的神经架构提供了一个有前景的方向。