We introduce the \textbf{B}i-Directional \textbf{S}parse \textbf{Hop}field Network (\textbf{BiSHop}), a novel end-to-end framework for deep tabular learning. BiSHop handles the two major challenges of deep tabular learning: non-rotationally invariant data structure and feature sparsity in tabular data. Our key motivation comes from the recent established connection between associative memory and attention mechanisms. Consequently, BiSHop uses a dual-component approach, sequentially processing data both column-wise and row-wise through two interconnected directional learning modules. Computationally, these modules house layers of generalized sparse modern Hopfield layers, a sparse extension of the modern Hopfield model with adaptable sparsity. Methodologically, BiSHop facilitates multi-scale representation learning, capturing both intra-feature and inter-feature interactions, with adaptive sparsity at each scale. Empirically, through experiments on diverse real-world datasets, we demonstrate that BiSHop surpasses current SOTA methods with significantly less HPO runs, marking it a robust solution for deep tabular learning.

翻译：本文提出**双向稀疏Hopfield网络（BiSHop）**，一种用于深度表格学习的新型端到端框架。BiSHop解决了深度表格学习面临的两大核心挑战：表格数据的非旋转不变数据结构与特征稀疏性。我们的核心动机源于近期关联记忆与注意力机制之间建立的理论联系。为此，BiSHop采用双组件架构，通过两个相互关联的定向学习模块，依次对数据进行列向与行向处理。在计算层面，这些模块包含多层广义稀疏现代Hopfield层——这是具有可调节稀疏性的现代Hopfield模型的稀疏扩展。在方法论层面，BiSHop支持多尺度表征学习，能在每个尺度上以自适应稀疏性捕获特征内与特征间交互作用。通过在多领域真实数据集上的实验验证，我们证明BiSHop以显著更少的超参数优化次数超越当前SOTA方法，标志着其为深度表格学习提供了稳健的解决方案。