Recently a million of biological neurons (BNN) has turned out better from modern RL methods in playing Pong~\cite{RL}, reminding they are still qualitatively superior e.g. in learning, flexibility and robustness - suggesting to try to improve current artificial e.g. MLP/KAN for better agreement with biological. There is proposed extension of KAN approach to neurons containing model of local joint distribution: $ρ(\mathbf{x})=\sum_{\mathbf{j}\in B} a_\mathbf{j} f_\mathbf{j}(\mathbf{x})$ for $\mathbf{x} \in [0,1]^d$, adding interpretation and information flow control to KAN, and allowing to gradually add missing 3 basic properties of biological: 1) biological axons propagate in both directions~\cite{axon}, while current artificial are focused on unidirectional propagation - joint distribution neurons can repair by substituting some variables to get conditional values/distributions for the remaining. 2) Animals show risk avoidance~\cite{risk} requiring to process variance, and generally real world rather needs probabilistic models - the proposed can predict and propagate also distributions as vectors of moments: (expected value, variance) or higher. 3) biological neurons require local training, and beside backpropagation, the proposed allows many additional ways, like direct training, through tensor decomposition, or finally local and promising: information bottleneck. Proposed approach is very general, can be also used as extension of softmax in embeddings of e.g. transformer, suggesting interpretation that features are mixed moments of joint density of real-world properties.

翻译：近期，一项包含百万级生物神经元（BNN）的研究在《Pong》游戏任务中表现优于现代强化学习方法（RL）~\cite{RL}，这提醒我们生物神经元在学习能力、灵活性和鲁棒性等方面仍具有质的优势——这启示我们尝试改进当前的人工神经元（如MLP/KAN）以更好地与生物学特性对齐。本文提出对KAN方法的扩展，使其神经元包含局部联合分布模型：对于$\mathbf{x} \in [0,1]^d$，有$ρ(\mathbf{x})=\sum_{\mathbf{j}\in B} a_\mathbf{j} f_\mathbf{j}(\mathbf{x})$。该扩展为KAN增加了可解释性和信息流控制能力，并允许逐步弥补当前人工神经元所缺失的三个基本生物学特性：1）生物轴突可双向传播信号~\cite{axon}，而当前人工神经元主要关注单向传播——联合分布神经元可通过替换部分变量来获取剩余变量的条件值/分布，从而修复此缺陷。2）动物表现出风险规避行为~\cite{risk}，这要求处理方差信息；一般而言，现实世界更需要概率模型——所提出的神经元能够以矩向量（如期望值、方差或更高阶矩）的形式预测和传播分布。3）生物神经元需要局部训练，除反向传播外，所提出的方法允许许多额外的训练方式，例如直接训练、通过张量分解进行训练，或最终采用局部且前景广阔的信息瓶颈方法。所提出的方法具有高度通用性，也可用作嵌入层（例如Transformer中的嵌入层）中softmax的扩展，这暗示了一种解释：特征可视为现实世界属性的联合密度的混合矩。