高级检索

多级正例约束的无监督句子表示

Multi-Level Positive Sample Constrained Unsupervised Sentence Representation

  • 摘要: 无监督句子表示旨在从大规模的文本数据中学习句子嵌入的空间结构。当前的工作使用对比学习来约束不同句向量之间的余弦距离,但对比学习的损失函数仅仅考虑正负例两种距离关系,与表示语义相似度的连续实数之间存在冲突。因此该文利用依存句法树以及关键短语抽取算法,对每一个句子构建了相似程度不同的正例,并引入了最大熵列表排序损失,学习不同正例之间的距离。在语义文本相似度数据集上的实验表明,该文提出的模型有效缓解了对比学习损失函数的离散性与语义相似度的连续性之间的冲突,并在多个预训练编码器上取得了有效的性能提升,平均斯皮尔曼系数为79.79%,较对比的最优无监督句子表示基线有1.03%的提升。

     

    Abstract: Unsupervised sentence representation aims to learn the spatial structure of sentence embeddings from large-scale text data. The current work uses contrastive learning to constrain the cosine distance between different sentence vectors via the loss function built upon positive and negative examples. To better capture the continuous real number of semantic similarity, this paper proposes the maximum entropy list sorting loss to learn the distance between different positive examples by utilizing the dependency syntax tree and the key phrase extraction algorithm to construct positive examples with different degrees of similarity for each sentence. Experiments on the Semantic Text Similarity datasets show that the proposed method achieves an average Spearman coefficient of 79.79%, which is 1.03% higher than the current state-of-the-art baselines.

     

/

返回文章
返回