Over the past two decades, recommendation systems (RSs) have used machine learning (ML) solutions to recommend items, e.g., movies, books, and restaurants, to clients of a business or an online platform. Recipe recommendation, however, has not yet received much attention compared to those applications. We introduce RECipe as a multi-purpose recipe recommendation framework with a multi-modal knowledge graph (MMKG) backbone. The motivation behind RECipe is to go beyond (deep) neural collaborative filtering (NCF) by recommending recipes to users when they query in natural language or by providing an image. RECipe consists of 3 subsystems: (1) behavior-based recommender, (2) review-based recommender, and (3) image-based recommender. Each subsystem relies on the embedding representations of entities and relations in the graph. We first obtain (pre-trained) embedding representations of textual entities, such as reviews or ingredients, from a fine-tuned model of Microsoft's MPNet. We initialize the weights of the entities with these embeddings to train our knowledge graph embedding (KGE) model. For the visual component, i.e., recipe images, we develop a KGE-Guided variational autoencoder (KG-VAE) to learn the distribution of images and their latent representations. Once KGE and KG-VAE models are fully trained, we use them as a multi-purpose recommendation framework. For benchmarking, we created two knowledge graphs (KGs) from public datasets on Kaggle for recipe recommendation. Our experiments show that the KGE models have comparable performance to the neural solutions. We also present pre-trained NLP embeddings to address important applications such as zero-shot inference for new users (or the cold start problem) and conditional recommendation with respect to recipe categories. We eventually demonstrate the application of RECipe in a multi-purpose recommendation setting.

翻译：摘要：过去二十年，推荐系统（RSs）利用机器学习（ML）解决方案，为商业或在线平台的客户推荐电影、书籍和餐厅等物品。然而，相较于这些应用，食谱推荐尚未获得足够关注。我们提出RECipe——一种以多模态知识图谱（MMKG）为骨干的多用途食谱推荐框架。RECipe的动机在于超越（深度）神经协同过滤（NCF），通过自然语言查询或提供图像的方式向用户推荐食谱。RECipe包含三个子系统：（1）基于行为的推荐器，（2）基于评论的推荐器，以及（3）基于图像的推荐器。每个子系统依赖图中实体和关系的嵌入表示。我们首先从微软MPNet的微调模型中获取文本实体（如评论或食材）的（预训练）嵌入表示，并以此初始化实体权重，训练知识图谱嵌入（KGE）模型。对于视觉组件（即食谱图像），我们开发了知识图谱引导的变分自编码器（KG-VAE），以学习图像分布及其潜在表示。当KGE和KG-VAE模型完全训练后，我们将其用作多用途推荐框架。为进行基准测试，我们从Kaggle上的公开数据集构建了两个食谱推荐知识图谱（KGs）。实验表明，KGE模型的性能与神经解决方案相当。我们还提出了预训练的NLP嵌入，以解决重要应用场景，例如对新用户的零样本推理（或冷启动问题）以及基于食谱类别的条件推荐。最终，我们展示了RECipe在多用途推荐环境中的应用。