问句复述识别旨在识别两个自然问句是否语义一致。目前,基于表示学习和深度神经网络架构的复述识别技术已取得较好效果。但是,这类方法往往面临复杂度较高且训练难度较大的瓶颈。针对这一问题,该文提出一种快速的多卷积自交互匹配方法。该方法融合了多种句子特征和词义特征,并由此形成分布式表示。在此基础上,这一方法利用卷积神经网络获取短语级的句子向量表示,并利用自交互融合技术将词级与短语级的向量表示进行充分融合,借以获得多粒度句子向量表示。该文将这一方法应用于判定自然语句是否互为复述的二元分类任务中,利用Quora标准问句复述识别语料进行测试。实验结果证明,在不引入外部数据的情况下,该文所提方法与基于双向多视角匹配的基准模型具有可比的性能,但在空间复杂度上具有较高的优越性,并且获得更快训练速度。具体地,该方法训练所需的物理显存比基准模型方法下降80%,训练迭代速度快19倍。
Abstract
Question paraphrase identification aims to identify whether two natural questions are semantic consistency. At present, paraphrase identification technology based on representation learning and deep neural network architecture has achieved good results. However, these methods often face bottlenecks with high complexity and difficulty in training. To tackle the problem, this paper proposes a fast method named multi-convolution self-interaction match (MCSM) model. This method combines multiple sentence features with word sense features to form a distributed representation. Then it utilizes convolution neural networks to capture phrase-level sentence representation. A self-interaction fusion technology is employed to obtain multi-granularity sentence vector representation, which can fully integrate word-level and phrase-level feature vector representation. Experimented on the Quora standard paraphrase identification corpus, the proposed method has comparable performance to the benchmark model based on bilateral multi-perspective match without external data. But has a training speed of its 19 times faster than the baseline, and the memory required is reduced by 80%.
关键词
复述识别 /
多卷积交互 /
效率
{{custom_keyword}} /
Key words
paraphrase identification /
multi convolution interaction /
efficiency
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Lei T, Joshi H, Barzilay R, et al. Semi-supervised Question Retrieval with Gated Convolutions[C]//Proceedings of NAACL2016,2016:1279-1289.
[2] Dipanjan Das N A S. Paraphrase identification as probabilistic quasi-synchronous recognition[C]//Proceedings of the Joint Conference of the 47th Amual Meeting of the Acl & the 4th International Joint Conference on Natural Language Processing of the Afnlp: 2009.
[3] Wang Z, Hamza W, Florian R. Bilateral multi-perspective matching for natural language sentences[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. AAAI Press, 2017:4144-4150.
[4] Tomar G S, Duque T, Tāckstr?m O, et al. Neural paraphrase identification of questions with noisy pretraining[C]//Proceedings of the 1st Workshop on Subword and Character Level Models in NLP, 2017:142-147.
[5] Pang L, Lan Y, Guo J, et al. Text matching as image recognition[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. AAAI Press, 2016:2793-2799.
[6] Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 2014, 1:655-665.
[7] Wan S, Dras M, Dale R, et al. Using dependency-based features to take the “para-fare” out of paraphrase[C]//Proceedings of the Australasian Language Technology Workshop 2006, 2006:131-138.
[8] Madnani N, Tetreault J, Chodorow M. Re-examining machine translation metrics for paraphrase identification[C]//Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2012:182-190.
[9] Wu D. Recognizing paraphrases and textual entailment using inversion transduction grammars[C]//Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment. Association for Computational Linguistics, 2005:25-30.
[10] Guo W, Diab M. Modeling sentences in the latent space[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2012:864-872.
[11] Neculoiu P, Versteegh M, Rotaru M. Learning text similarity with siamese recurrent networks[C]//Proceedings of the 1st Workshop on Representation Learning for NLP, 2016:148-157.
[12] Yu A W, Dohan D, Luong M T, et al. QANet: Combining local convolution with global self-attention for reading comprehension[J]. arXiv preprint arXiv: 1804.09541, 2018.
[13] Srivastava R K, Greff K, Schmidhuber J. Highway networks[J]. arXiv preprint arXiv: 1505.00387, 2015.
[14] Pennington J, Socher R, Manning C. Glove: Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014:1532-1543.
[15] Chen D, Fisch A, Weston J, et al. Reading Wikipedia to answer open-domain questions[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017, 1:1870-1879.
[16] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J].arXiv preprint arXiv: 1706.03762,2017.
[17] Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv: 1412.6980, 2014.
[18] Ruder S. An overview of gradient descent optimization algorithms[J]. arXiv preprint arXiv: 1609.04747, 2016.
[19] Fan Y, Pang L, Hou J P, et al. Matchzoo: A toolkit for deep text matching[J]. arXiv preprint arXiv: 1707.07270, 2017.
[20] Fader A, Zettlemoyer L, Etzioni O. Paraphrase-driven learning for open question answering[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 2013, 1:1608-1618.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61672367,61672368);国家重点研发计划(2017YFB1002104);江苏省研究生科研与实践创新计划项目(SJCX19_0926)
{{custom_fund}}
PDF(2967 KB)
