图片描述翻译是给定图片及图片在某一语言的描述,利用翻译技术为图片生成目标语言描述的任务。观察发现,不同图片表达的场景往往不同,对应的图片描述具有明显的主题差异性。因此,利用主题信息能够提升翻译效果。然而,图片描述的内容通常较短,无法有效反映其主题。针对该问题,该文提出了一种融合图片主题信息的图片描述翻译方法。对于任意的图片描述对,该方法首先借助相似图片检索技术从维基百科图片库中检索与源图片相似的目标图片,进而利用包含目标图片的文档学习源图片的主题表示。最终,利用训练集中所有图片描述对的主题表示重新学习并获取适应主题的翻译模型。实验结果表明,借助相似图片获取信息量更为丰富的描述文本,并利用文本的主题信息强化翻译模型的方法,能够提高现有统计机器翻译系统的性能,在WMT16测试集上进行的评测显示,翻译质量的BLEU值提升了0.74个百分点。
Abstract
Image Description Translation takes a source language description and translates it into the target language, where this process can be supported by information from the image. Observations show that different images often express different scenes, the corresponding image description has obvious differences in topic distributions. This paper presents an image description translation method integrating the topic information of the image. For a pair of image and its descriptions, the method retrieves similar images from wiki, and then use the documents of the retrieved images to learn topic distributions. Finally, we use topic distributions of all training images and their descriptions to relearn the topic distribution, and get the translation model of topic adaptation. Our experimental results on the WMT16 test set show an improvement of 0.74 BLEU point over baseline.
关键词
图片描述翻译 /
主题差异性 /
图片检索
{{custom_keyword}} /
Key words
image description translation /
topic difference /
image retrieval
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Iacer Calixto,Desmond Elliott,Stella Frank.DCU-UvA multimodal MT system report[C]//Proceedings of Association for Computational Linguistics(ACL),2016:634-638.
[2] Po-Yao Huang,Frederick Liu,Sz-Rung Shiang,et al.Attention-based multimodal neural machine translation[C]//Proceedings of Association for Computational Linguistics(ACL),2016:639-645.
[3] Sergio Rodriguez Guasch,Marta R Costa-jussa.WMT 2016 multimodal translation system description based on bidirectional recurrent neural networks with double-embeddings[C]//Proceedings of Association for Computational Linguistics(ACL),2016:655-659.
[4] Bing Zhao,Eric P Xing.BiTAM:Bilingual topic admixture models for word alignment[C]//Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics,2006:969-976.
[5] Yik-Cheung Tam,Ian R Lane,Tanja Schultz.Bilingual lsa-based adaptation for statistical machine translation[J].Machine Translation,2007,21(4):187-207.
[6] Zhengxian Gong,Yu Zhang,Guodong Zhou.Statistical machine translation based on LDA[C]//Proceedings of the 4th International.IEEE,2010:286-290.
[7] Jinsong Su,Hua Wu,Haifeng Wang,et al.Translation model adaptation for statistical machine translation with monolingual topic information[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics,Association for Computational Linguistics,2012:459-468.
[8] Koehn P,Och F J,Marcu D.Statistical phrase-based translation[C]//Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology(Volume 1).Association for Computational Linguistics,2003:48-54.
[9] Xinyan Xiao,Deyi Xiong,Min Zhang,et al.A topic similarity model for hierarchical phrase-based translation[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics(ACL),2012:750-758.
[10] Jia Y,Shelhamer E,Donahue J,et al.Caffe:Convolutional architecture for fast feature embedding[C]//Proceedings of the 22nd ACM International Conference on Multimedia.ACM,2014:675-678.
[11] David M Blei,Andrew Y Ng,Michael I Jordan.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3(Jan):993-1022.
[12] Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[J].arXiv preprint.2015,1-14.
[13] Tong Xiao,Jingbo Zhu,Hao Zhang,et al.NiuTrans:An open source toolkit for phrase-based and syntax-based machine translation[C]//Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics(ACL),2012:19-24.
[14] Franz Josef Och,Hermann Ney.A systematic comparison of various statistical alignment models[J].Computational Linguistics.,2003,29(Jan):19-51.
[15] Andreas Stolcke.SRILM:An extensible language modeling toolkit[C]//Proceedings of the International Conference on Spoken Language Processing(ICSLP),2002:901-904.
[16] Franz Josef Och.Minimum error rate training in statistical machine translation[C]//Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics(ACL),2003:160-167.
[17] Kishore Papineni,Salim Roukos,Todd Ward,et al.BLEU:A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics(ACL),2002:311-318.
[18] P Koehn,FJ Och,D Marcu.Statistical phrase-based transaltion[C]//Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (Volume 1).Association for Computational Linguistics,2003:48-54.
[19] Zhang D,Li M,Li C H,et al.Phrase reordering model integrating syntactic knowledge for SMT[C]//Proceedings of the 2007 EMNLP-CoNLL,2007:533-540.
[20] Zens R,Ney H.Improvements in phrase-based statistical machine translation[C]//Proceedings of the 2004 HLT-NAACL,2004:257-264.
[21] Le Q,Mikolov T.Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on Machine Learning (ICML-14),2014:1188-1196.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61373097, 61672368, 61672367, 61331011);江苏省科技计划(SBK2015022101);教育部—中国移动科研基金(MCM20150602)
{{custom_fund}}
PDF(2115 KB)
