In multimodal-aware recommendation, the extraction of meaningful multimodal features is at the basis of high-quality recommendations. Generally, each recommendation framework implements its multimodal extraction procedures with specific strategies and tools. This is limiting for two reasons: (i) different extraction strategies do not ease the interdependence among multimodal recommendation frameworks; thus, they cannot be efficiently and fairly compared; (ii) given the large plethora of pre-trained deep learning models made available by different open source tools, model designers do not have access to shared interfaces to extract features. Motivated by the outlined aspects, we propose \framework, a unified framework for the extraction of multimodal features in recommendation. By integrating three widely-adopted deep learning libraries as backends, namely, TensorFlow, PyTorch, and Transformers, we provide a shared interface to extract and process features where each backend's specific methods are abstracted to the end user. Noteworthy, the extraction pipeline is easily configurable with a YAML-based file where the user can specify, for each modality, the list of models (and their specific backends/parameters) to perform the extraction. Finally, to make \framework accessible to the community, we build a public Docker image equipped with a ready-to-use CUDA environment and propose three demos to test its functionalities for different scenarios and tasks. The GitHub repository and the documentation are accessible at this link: https://github.com/sisinflab/Ducho.
翻译:在多模态感知推荐中,提取有意义的多模态特征是高质量推荐的基础。通常,每个推荐框架采用特定策略与工具实现其多模态提取流程,这存在两方面的局限性:(i)不同的提取策略阻碍了多模态推荐框架间的相互依赖,导致无法进行高效且公平的比较;(ii)面对不同开源工具提供的大量预训练深度学习模型,模型设计者无法通过统一接口提取特征。针对上述问题,本文提出 **Ducho**——一个用于推荐任务的多模态特征提取统一框架。通过整合TensorFlow、PyTorch和Transformers三种主流深度学习库作为后端,我们构建了共享接口用于提取和处理特征,将各后端特有的方法对终端用户透明化。值得注意的是,提取流程可通过基于YAML的配置文件轻松定制,用户可为每种模态指定模型列表(及其后端/参数)以执行特征提取。最后,为提升框架的可访问性,我们构建了配备即用CUDA环境的公开Docker镜像,并提供三个演示用例以测试其在不同场景和任务中的功能。GitHub仓库与文档可通过以下链接访问:https://github.com/sisinflab/Ducho。