Due to the usefulness in data enrichment for data analysis tasks, joinable table discovery has become an important operation in data lake management. Existing approaches target equi-joins, the most common way of combining tables for creating a unified view, or semantic joins, which tolerate misspellings and different formats to deliver more join results. They are either exact solutions whose running time is linear in the sizes of query column and target table repository or approximate solutions lacking precision. In this paper, we propose Deepjoin, a deep learning model for accurate and efficient joinable table discovery. Our solution is an embedding-based retrieval, which employs a pre-trained language model (PLM) and is designed as one framework serving both equi- and semantic joins. We propose a set of contextualization options to transform column contents to a text sequence. The PLM reads the sequence and is fine-tuned to embed columns to vectors such that columns are expected to be joinable if they are close to each other in the vector space. Since the output of the PLM is fixed in length, the subsequent search procedure becomes independent of the column size. With a state-of-the-art approximate nearest neighbor search algorithm, the search time is logarithmic in the repository size. To train the model, we devise the techniques for preparing training data as well as data augmentation. The experiments on real datasets demonstrate that by training on a small subset of a corpus, Deepjoin generalizes to large datasets and its precision consistently outperforms other approximate solutions'. Deepjoin is even more accurate than an exact solution to semantic joins when evaluated with labels from experts. Moreover, when equipped with a GPU, Deepjoin is up to two orders of magnitude faster than existing solutions.

翻译：由于其在数据分析任务中数据增强的实用性，可连接表发现已成为数据湖管理中的关键操作。现有方法针对等值连接（最常用的表合并方式以创建统一视图）或语义连接（容忍拼写错误及不同格式以提供更多连接结果）。这些方法要么是精确解，其运行时间与查询列和目标表库的大小呈线性关系，要么是缺乏精度的近似解。本文提出Deepjoin，一种用于准确高效可连接表发现的深度学习模型。我们的解决方案是基于嵌入的检索，利用预训练语言模型（PLM）并设计为同时服务于等值连接和语义连接的统一框架。我们提出一组情境化选项将列内容转换为文本序列，由PLM读取序列并通过微调将列嵌入为向量，使得在向量空间中彼此接近的列预期可连接。由于PLM输出长度固定，后续搜索过程与列大小无关。结合最先进的近似最近邻搜索算法，搜索时间与存储库规模呈对数关系。为训练模型，我们设计了训练数据准备及数据增强技术。在真实数据集上的实验表明，通过在小规模语料子集上训练，Deepjoin可泛化至大型数据集，其精度持续优于其他近似解。当基于专家标注评估时，Deepjoin在语义连接上的准确性甚至超过精确解。此外，配备GPU时，Deepjoin的速度比现有解决方案快两个数量级。