Learning expressive representations for high-dimensional yet sparse features has been a longstanding problem in information retrieval. Though recent deep learning methods can partially solve the problem, they often fail to handle the numerous sparse features, particularly those tail feature values with infrequent occurrences in the training data. Worse still, existing methods cannot explicitly leverage the correlations among different instances to help further improve the representation learning on sparse features since such relational prior knowledge is not provided. To address these challenges, in this paper, we tackle the problem of representation learning on feature-sparse data from a graph learning perspective. Specifically, we propose to model the sparse features of different instances using hypergraphs where each node represents a data instance and each hyperedge denotes a distinct feature value. By passing messages on the constructed hypergraphs based on our Hypergraph Transformer (HyperFormer), the learned feature representations capture not only the correlations among different instances but also the correlations among features. Our experiments demonstrate that the proposed approach can effectively improve feature representation learning on sparse features.

翻译：为高维稀疏特征学习表达性表示一直是信息检索领域中的长期难题。尽管近期深度学习方法能部分解决这一问题，但它们往往难以处理大量稀疏特征，尤其是那些在训练数据中低频出现的尾部特征值。更糟的是，现有方法无法显式利用不同实例间的相关性来进一步改进稀疏特征表示学习，因为这种关系先验知识并未提供。为应对这些挑战，本文从图学习视角处理特征稀疏数据的表示学习问题。具体而言，我们提出使用超图对不同实例的稀疏特征进行建模，其中每个节点代表一个数据实例，每条超边代表一个不同的特征值。通过基于我们提出的超图Transformer（HyperFormer）在构建的超图上传递消息，学习到的特征表示不仅捕捉了不同实例间的相关性，还捕捉了特征间的相关性。我们的实验表明，所提方法能有效改善稀疏特征上的特征表示学习。