It has been an open question in deep learning if fault-tolerant computation is possible: can arbitrarily reliable computation be achieved using only unreliable neurons? In the grid cells of the mammalian cortex, analog error correction codes have been observed to protect states against neural spiking noise, but their role in information processing is unclear. Here, we use these biological codes to show that a universal fault-tolerant neural network can be achieved if the faultiness of each neuron lies below a sharp threshold; moreover, we find that noisy biological neurons fall below this threshold. The discovery of a phase transition from faulty to fault-tolerant neural computation suggests a mechanism for reliable computation in the cortex and opens a path towards understanding noisy analog systems relevant to artificial intelligence and neuromorphic computing.
翻译:在深层学习中,如果可以进行容错计算,这是个开放的问题:只能使用不可靠的神经元,才能实现任意可靠的计算?在哺乳动物皮层的电网细胞中,观察到了类似的错误校正代码,以保护国家免受神经突飞猛进的噪音,但它们在信息处理中的作用并不明确。在这里,我们使用这些生物代码来表明,如果每个神经元的缺陷都低于尖锐的临界值,那么就能够实现普遍的容错神经网络;此外,我们发现吵闹的生物神经元低于这一临界值。 发现从错误到容错错的神经计算的阶段过渡意味着在皮层中建立可靠的计算机制,并开辟了理解与人工智能和神经形态计算有关的吵闹的模拟系统的途径。