Fully test-time adaptation aims to adapt the network model based on sequential analysis of input samples during the inference stage to address the cross-domain performance degradation problem of deep neural networks. We take inspiration from the biological plausibility learning where the neuron responses are tuned based on a local synapse-change procedure and activated by competitive lateral inhibition rules. Based on these feed-forward learning rules, we design a soft Hebbian learning process which provides an unsupervised and effective mechanism for online adaptation. We observe that the performance of this feed-forward Hebbian learning for fully test-time adaptation can be significantly improved by incorporating a feedback neuro-modulation layer. It is able to fine-tune the neuron responses based on the external feedback generated by the error back-propagation from the top inference layers. This leads to our proposed neuro-modulated Hebbian learning (NHL) method for fully test-time adaptation. With the unsupervised feed-forward soft Hebbian learning being combined with a learned neuro-modulator to capture feedback from external responses, the source model can be effectively adapted during the testing process. Experimental results on benchmark datasets demonstrate that our proposed method can significantly improve the adaptation performance of network models and outperforms existing state-of-the-art methods.

翻译：完全测试时适应旨在通过推理阶段对输入样本进行序列分析来调整网络模型，以解决深度神经网络跨域性能退化问题。我们借鉴生物合理性学习机制，其中神经元响应基于局部突触变化过程进行调整，并通过竞争性侧向抑制规则激活。基于这些前馈学习规则，我们设计了一种软Hebbian学习过程，为在线自适应提供了一种无监督且有效的机制。我们观察到，通过引入反馈神经调制层，这种前馈Hebbian学习在完全测试时适应中的性能可显著提升。该层能够基于顶层推理层误差反向传播产生的外部反馈，对神经元响应进行微调。这促使我们提出用于完全测试时适应的神经调制Hebbian学习（NHL）方法。通过将无监督前馈软Hebbian学习与捕获外部响应反馈的学习型神经调制器相结合，源模型可在测试过程中有效适配。基准数据集上的实验结果表明，我们提出的方法能显著提升网络模型的适配性能，并优于现有最先进方法。