We present UNSEE: Unsupervised Non-Contrastive Sentence Embeddings, a novel approach that outperforms SimCSE in the Massive Text Embedding benchmark. Our exploration begins by addressing the challenge of representation collapse, a phenomenon observed when contrastive objectives in SimCSE are replaced with non-contrastive objectives. To counter this issue, we propose a straightforward solution known as the target network, effectively mitigating representation collapse. The introduction of the target network allows us to leverage non-contrastive objectives, maintaining training stability while achieving performance improvements comparable to contrastive objectives. Our method has achieved peak performance in non-contrastive sentence embeddings through meticulous fine-tuning and optimization. This comprehensive effort has yielded superior sentence representation models, showcasing the effectiveness of our approach.

翻译：我们提出UNSEE：无监督非对比句嵌入，这是一种在大规模文本嵌入基准测试中优于SimCSE的新方法。我们从解决表示崩溃这一挑战入手展开探索——当SimCSE中的对比目标被非对比目标替代时，会出现这一现象。针对该问题，我们提出一种称为目标网络的直接解决方案，有效缓解了表示崩溃。引入目标网络使我们能够利用非对比目标，在保持训练稳定性的同时，实现与对比目标相媲美的性能提升。通过精细的调优与优化，我们的方法在非对比句嵌入任务中达到了峰值性能。这一全面工作产出了更优的句子表示模型，充分证明了我们方法的有效性。