Contrastive Learning for Unsupervised Sentence Embedding with False Negative Calibration

Contrastive Learning, a transformative approach to the embedding of unsupervised sentences, fundamentally works to amplify similarity within positive samples and suppress it amongst negative ones. However, an obscure issue associated with Contrastive Learning is the occurrence of False Negatives, which treat similar samples as negative samples that will hurt the semantics of the sentence embedding. To address it, we propose a framework called FNC (False Negative Calibration) to alleviate the influence of false negatives. Our approach has two strategies to amplify the effect, i.e. false negative elimination and reuse. Specifically, in the training process, our method eliminates false negatives by clustering and comparing the semantic similarity. Next, we reuse those eliminated false negatives to reconstruct new positive pairs to boost contrastive learning performance. Our experiments on seven semantic textual similarity tasks demonstrate that our approach is more effective than competitive baselines.