Self-supervised representation learning methods have achieved significant success in computer vision and natural language processing, where data samples exhibit explicit spatial or semantic dependencies. However, applying these methods to tabular data is challenging due to the less pronounced dependencies among data samples. In this paper, we address this limitation by introducing SwitchTab, a novel self-supervised method specifically designed to capture latent dependencies in tabular data. SwitchTab leverages an asymmetric encoder-decoder framework to decouple mutual and salient features among data pairs, resulting in more representative embeddings. These embeddings, in turn, contribute to better decision boundaries and lead to improved results in downstream tasks. To validate the effectiveness of SwitchTab, we conduct extensive experiments across various domains involving tabular data. The results showcase superior performance in end-to-end prediction tasks with fine-tuning. Moreover, we demonstrate that pre-trained salient embeddings can be utilized as plug-and-play features to enhance the performance of various traditional classification methods (e.g., Logistic Regression, XGBoost, etc.). Lastly, we highlight the capability of SwitchTab to create explainable representations through visualization of decoupled mutual and salient features in the latent space.

翻译：自监督表示学习方法在计算机视觉和自然语言处理领域取得了显著成功，这些领域中的数据样本表现出明确的空间或语义依赖性。然而，将这些方法应用于表格数据时，由于数据样本间的依赖性较弱，面临着挑战。本文通过引入SwitchTab——一种专门设计用于捕捉表格数据中潜在依赖关系的新型自监督方法，来应对这一局限。SwitchTab利用非对称编码器-解码器框架来解耦数据对中的共性与显著性特征，从而生成更具代表性的嵌入表示。这些嵌入进而有助于形成更优的决策边界，并在下游任务中取得更佳结果。为验证SwitchTab的有效性，我们在涉及表格数据的多个领域进行了广泛实验。结果表明，在微调后的端到端预测任务中，该方法展现出优越的性能。此外，我们证明了预训练的显著性嵌入可作为即插即用特征，用于提升各类传统分类方法（如逻辑回归、XGBoost等）的表现。最后，我们通过可视化潜空间中解耦后的共性与显著性特征，突显了SwitchTab创建可解释表示的能力。