Since their inception, artificial neural networks have relied on manually designed architectures and inductive biases to better adapt to data and tasks. With the rise of deep learning and the expansion of parameter spaces, they have begun to exhibit brain-like functional behaviors. Nevertheless, artificial neural networks remain fundamentally different from biological neural systems in structural organization, learning mechanisms, and evolutionary pathways. From the perspective of neuroscience, we rethink the formation and evolution of intelligence and proposes a new neural network paradigm, Brain-like Neural Network (BNN). We further present the first instantiation of a BNN termed LuminaNet that operates without convolutions or self-attention and is capable of autonomously modifying its architecture. We conduct extensive experiments demonstrating that LuminaNet can achieve self-evolution through dynamic architectural changes. On the CIFAR-10, LuminaNet achieves top-1 accuracy improvements of 11.19%, 5.46% over LeNet-5 and AlexNet, respectively, outperforming MLP-Mixer, ResMLP, and DeiT-Tiny among MLP/ViT architectures. On the TinyStories text generation task, LuminaNet attains a perplexity of 8.4, comparable to a single-layer GPT-2-style Transformer, while reducing computational cost by approximately 25% and peak memory usage by nearly 50%. Code and interactive structures are available at https://github.com/aaroncomo/LuminaNet.
翻译:自诞生以来,人工神经网络一直依赖于人工设计的架构和归纳偏置,以更好地适应数据和任务。随着深度学习的兴起和参数空间的扩展,它们开始展现出类脑的功能行为。然而,人工神经网络在结构组织、学习机制和演化路径上,与生物神经系统仍存在根本性差异。从神经科学的角度出发,我们重新思考智能的形成与演化,并提出一种新的神经网络范式——类脑神经网络(BNN)。我们进一步提出了首个BNN实例,命名为LuminaNet,该网络无需卷积或自注意力机制即可运行,并能够自主修改其架构。我们进行了大量实验,证明LuminaNet能够通过动态架构变化实现自我演化。在CIFAR-10数据集上,LuminaNet的top-1准确率分别比LeNet-5和AlexNet提高了11.19%和5.46%,在MLP/ViT架构中超越了MLP-Mixer、ResMLP和DeiT-Tiny。在TinyStories文本生成任务中,LuminaNet达到了8.4的困惑度,与单层GPT-2风格的Transformer相当,同时计算成本降低了约25%,峰值内存使用量减少了近50%。代码和交互结构可在https://github.com/aaroncomo/LuminaNet获取。