We present UNSEE: Unsupervised Non-Contrastive Sentence Embeddings, a novel approach that outperforms SimCSE in the Massive Text Embedding benchmark. Our exploration begins by addressing the challenge of representation collapse, a phenomenon observed when contrastive objectives in SimCSE are replaced with non-contrastive objectives. To counter this issue, we propose a straightforward solution known as the target network, effectively mitigating representation collapse. The introduction of the target network allows us to leverage non-contrastive objectives, maintaining training stability while achieving performance improvements comparable to contrastive objectives. Our method has achieved peak performance in non-contrastive sentence embeddings through meticulous fine-tuning and optimization. This comprehensive effort has yielded superior sentence representation models, showcasing the effectiveness of our approach.

翻译：我们提出UNSEE：一种无监督非对比句子嵌入方法，该新方法在大规模文本嵌入基准测试中表现优于SimCSE。我们的探索始于解决表征坍塌这一挑战——该现象出现在用非对比目标替代SimCSE中对比目标时。为应对此问题，我们提出了一种称为目标网络的直接解决方案，有效缓解了表征坍塌。目标网络的引入使我们能够利用非对比目标，在保持训练稳定性的同时，实现与对比目标相当的性能提升。通过精细调优与优化，我们的方法在非对比句子嵌入中达到了峰值性能。这一全面工作产生了更优的句子表示模型，充分证明了我们方法的有效性。