Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art with a high barrier to entry. While many components are familiar, successfully training a SSL method involves a dizzying set of choices from the pretext tasks to training hyper-parameters. Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. We hope to empower the curious researcher to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be.
翻译:自监督学习,被誉为智能的暗物质,是推动机器学习发展的有前途路径。然而,如同烹饪一般,训练自监督学习方法是一项门槛极高的微妙艺术。尽管许多组件并不陌生,但成功训练自监督学习方法涉及从预训练任务到训练超参数的令人眼花缭乱的选择。我们的目标是降低进入自监督学习研究的门槛,以烹饪食谱的风格阐述基础知识和最新的自监督学习配方。我们希望能赋予好奇的研究者以能力,使其能够在方法领域中导航,理解各种调节参数的作用,并掌握探索自监督学习之美味的实践技能。