迁移学习在一定程度上减轻了目标域的数据稀疏问题对泛化能力的影响,然而泛化能力的提高仍然受到负迁移等问题的影响。为了解决负迁移问题,该文提出使用源域结构的文本语料的信息粒化方法,用区间信息粒表示出源域数据集的结构对数据集中统计量的影响。然后提出区间二型模糊隐马尔可夫模型(Interval Type-2 fuzzy Hidden Markov Model, IHMM) 以处理区间信息粒。给出了IHMM的构建方法和去模糊化方法。在文本的词性标注任务中进行了多个实验,可以证实利用源域结构信息的粒迁移学习方法避免了负迁移,提高了模型的泛化能力。
Abstract
Transfer learning alleviates the data sparseness issue to some extent, but the generalization capacity is still hindered by negative-transfer problem. To address this issue, we propose an information granulation method for text corpora based on source domain structure. Interval granules are employed to express the influence of source domain structure on statistics of the dataset. We further design an Interval Type-2 fuzzy Hidden Markov Model (IHMM) to deal with the interval granules. Experiments on part-of-speech tagging proves that the proposed method avoids negative-transfer and improves generalization capacity.
关键词
迁移学习 /
粒计算 /
区间信息粒 /
词性标注
{{custom_keyword}} /
Key words
transfer learning /
granular computing /
interval granules /
part-of-speech tagging
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Pan S J, Yang Q. A Survey on Transfer Learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[2] Pedrycz W. Granular computing: analysis and design of intelligent systems[M]. CRC press, 2013.
[3] Pedrycz W, Russo B, Succi G. Knowledge transfer in system modeling and its realization through an optimal allocation of information granularity[J]. Applied Soft Computing, 2012, 12(8): 1985-1995.
[4] Rabiner L. A tutorial on hidden Markov models and selected applications in speech recognition[J]. Readings in Spoech Recognition, 1990, 77(2): 267-296.
[5] Walder C J, Kootsookos B C, Peter J. andLovell. Towards a Maximum Entropy Method for Estimating HMM Parameters[C]//Proceedings of INTERSPEECH. 2003: 45-49.
[6] Liu J, Yu K, Zhang Y, et al. Training Conditional Random Fields Using Transfer Learning for Gesture Recognition[C]//Proceedings of IEEE International Conference on Data Mining. 2010: 314-323.
[7] Sutton C, McCallum A. Composition of conditional random fields for transfer learning[C]//Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. 2005: 748-754.
[8] Brants T. TnT: a statistical part-of-speech tagger[C]//Proceedings of the Sixth Conference on Applied Natural Language Processing. 2000: 224-231.
[9] Ait-Mohand K, Paquet T, Ragot N. Combining structure and parameter adaptation of HMMs for printed text recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(9): 1716-1732.
[10] Kim N S, Sung J S, Hong D H. Factored MLLR Adaptation[J]. Signal Processing Letters, 2011, 18(2): 99-102.
[11] 郭虎升, 王文剑. 动态粒度支持向量回归机[J]. 软件学报, 2013, 24(11): 2535-2547.
[12] 邱桃荣. 面向本体学习的粒计算方法研究[D]. 北京交通大学博士学位论文, 2009.
[13] Song M, Pedrycz W. Granular neural networks: concepts and development schemes[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(4): 542-553.
[14] 孟军. 相容粒计算模型及其数据挖掘研究[D]. 大连理工大学博士学位论文, 2012.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61472058, 61572102);辽宁省自然科学基金(201602195);中央高校自主基金(DC201502030202)
{{custom_fund}}
PDF(2888 KB)
孙世昶(1979—),博士,讲师,主要研究领域为机器学习与文本挖掘。E-mail: ssc@dlnu.edu.cn
林鸿飞(1962—),通信作者,博士,教授,主要研究领域为文本挖掘和信息检索。E-mail: lhf@dlut.edu.cn
孟佳娜 (1972—),博士,教授,主要研究领域为文本挖掘。E-mail: mjn@dlnu.edu.cn
