2019 Volume 33 Issue 6
Published: 24 June 2019
-
Select all|
-
Survey2019, 33(6): 1-11.The attention mechanism has gradually become one of the popular methods and research issues in deep learning. By improving the source language expression, it dynamically selects the related information of the source language in decoding, which greatly improves the insufficiency issue of the classic Encoder-Decoder framework. On the basis of the issues in the conventional Encoder-Decoder framework such as long-term memory limitation, interrelationships in sequence transformation, and output quality of model dynamic structure, this paper describes a varied aspects on attention mechanism, including the definition, the principle, the classification, state-of-the-art researches as well as the applications of attention mechanism in image recognition, speech recognition, and natural language processing. Meanwhile, this paper further discusses the multi-modal attention mechanism, evaluation mechanism of attention, interpretability of the model and integration of attention with the new model, providing new research issues and directions for the development of attention mechanism in deep learning.
-
Language Analysis and Calculation2019, 33(6): 12-17,34.There are at least six interpretations of wh-phrase zenme in Mandarin Chinese: universal implication, existential implication and interrogative implication (including 4 sub-division of situation, character, manner, reason). This paper proposes a rule-based approach to recognize zenme’s interpretations in different syntactic contexts. On the basis of the summarization of the syntactic contexts of the different complications, a word sense disambiguation model of zenme is proposed. The rule based method is validated and finally optimized by experiments.
-
Language Analysis and Calculation2019, 33(6): 18-26.Cross-lingual semantic textual similarity (STS) is to measure the degree of semantic similarity between texts in different languages. Most current neural network-based models use convolutional neural network to capture the local information of the text, without covering the semantic information between long-distance words in sentences. In this paper, we propose a neural network structure that combines gated convolutional neural networks and self-attention mechanism to obtain the local and global semantic correlations of cross-lingual text sentences, thus obtaining a better semantic representation of the sentences. The experimental results on several datasets of SemEval-2017 show that our model can capture the semantic similarity between sentences from different aspects, and outperforms the baselines based solely on neural network model.
-
Other Language in/around China2019, 33(6): 27-34.Cross-Language information retrieval is supposed to retrieve information in one language according to the queries of other languages. This paper uses cross language word vectors model to realize the mapping of Chinese query words to Mongolian query words. Three methods are proposed in this paper to perform mapping, namely Series, Series_opt and Cross_valid. These methods are used to map the Chinese queries as well as to select and sort the mapped words. Experimental conducted in the real environments show that our proposed algorithm can obtain improvement on Chinese-Mongolian information retrieval.
-
Other Language in/around China2019, 33(6): 35-41.To facilitate the structural analysis Tibetan and development of deeper of Tibetan knowledge, this paper proposes a method for Tibetan entity relation extraction based on optimized word vectors with GRU neural network model. In the training of the model, we apply the optimized word vector. In order to get a relatively good word vector model, we introduce the Tibetan syllable vector, the syllable position vector, part of speech vector and so on to further optimize the word vector. And we also select Tibetan lexical features and Tibetan sentence features. Experiments show that the proposed method achieves F1 value of 78.43%.
-
Other Language in/around China2019, 33(6): 42-49.Semantic understanding is an essential task in natural language understanding. Conventionally, grammar-rule-based approaches including lexical and sentence analysis are leveraged to parse the semantic meaning of given text. In this work, we present a new method to address Tibetan sentence semantic parsing via semantic chunking. The semantic chunking is modeled by Bi-LSTM and ID-CNN neural network , respectively. In experiments, the proposed model shows a remarkable performance, achieving the average F1 of 89% and 92%, respectively.
-
Other Language in/around China2019, 33(6): 50-56.This paper addresses the issue of automatic summarization for Korean texts and presents a novel Korean summarization (KKS) method based on key-noun extraction. We deem that Korean nouns mainly represent semantic information, while Korean predicates are more responsible for syntactic frame function. The experimental results show that the performance of our KKS algorithm is better than that of predicate-based one or all-word-based one, and the KKS algorithm can achieve the best performance in the Korean summarization task, which also proves the effectiveness of our assertion for the semantic function of Korean nouns.
-
Information Extraction and Text Mining2019, 33(6): 57-63.Event detection is the start point and impacts much on subsequent subtasks of information extraction. Towards better event detection, this paper proposes a sequence-to-sequence approach considering additional document-level information. It reduces the dependency on manual feature engineering by means of neural network, constructing an unified model combining local words, entities and global events co-occurrence in one document by the attention mechanism. Experimental results on LDC2017E02 corpus show that this method can effectively improve the performance of event recognition.
-
Information Extraction and Text Mining2019, 33(6): 64-71,79.The new event detection within topics is a less touched issue. The traditional new event detection methods mostly adopt the topic model-based method, which can not take into account subject information and semantic information. To deal with this issue, a method based on topical word embedding clustering is proposed for new event detection. This method firstly trains the TWE model on the pre-processed corpora and obtains the subject word vector; Secondly, the topic distribution is generated by K-means clustering of the topical word embedding; Again according to the new event detection process within topics, the new event detection problem is transformed into a new subtopic discovery problem. Finally, by using the acquired topic distribution, new events are detected for documents that arrive in chronological order. Experimental results show that the new method can take into account subject information and semantic information, and improve the performance of new event detection within topics effectively.
-
Information Extraction and Text Mining2019, 33(6): 72-79.Person name recognition tasks are often performed as part of the named entity recognition (NER) tasks, along with other types of entities. Currently, person name recognition method relies on the coverage of the training corpus for a particular type of person name, and the performance is significantly degraded when a new type of person name is encountered. To address this issue, we propose a method namesd Data Augmentation. In this method, we generate pseudo training data by replacing the common person name entities in training data with new specific types of entities. This method can effectively improve the recognition performance of the system for new types of person names. We propose a greedy representative subtype name selection algorithm which can select typical person name of a specific type. We conduct experiments on two test data sets: one is pseudo test data set based on the People's Daily data in 1998 and the other is manually labeled news data. The F1 measure of the recognition result is increased by at least 12% and 6%, respectively.
-
Information Extraction and Text Mining2019, 33(6): 80-87.Name entity recognition is a classical research issue in data mining community. To recognize the entities in multi-domain with fine-grained labels, we propose a method of utilizes web thesaurus to annotate web data automatically to acquire large-scale training corpus. To minimize the influence of the noises in training corpus, we design a two-phase entity recognition method. First, the entity’s domain label is obtained. After that, the context of each recognized entity is used to determine the fine-grained label for one entity. Experimental results demonstrate that the proposed method can obtain high accuracy on entity recognition in multiple domains.
-
Question Answering, Dialogue System and Machine Reading Comprehension2019, 33(6): 88-93.The question answering (QA) system based on medical KB has important research and application significance. Aimed at the primary liver cancer common in adults, this paper extracts related knowledge triples from the medical guides and SemMedDB to construct a KB of primary liver cancer. On this basis, a pipeline QA system is implemented. Firstly the system identifies the entity from the question. Then the sentence embedding is generated by combining TFIDF and the word embedding to select the most similar problem template. Finally the system retrieves the answer from the KB according to the semantics of the template and the entity in the question. The results show that, this system can effectively answer questions about drugs, diseases and symptoms related to primary liver cancer.
-
Sentiment Analysis and Social Computing2019, 33(6): 94-99.With the rapid development of artificial intelligence in recent years, the demand for commonsense know-ledge has become more and more urgent. As a part of commonsense knowledge, emotion commonsense knowledge is also an important aspect of affective computing. In view of the limitations in the structure and content of the emotion dictionary, we present an emotion commonsense library of binary structure. To construct the library, the set of emotion commonsense knowledge candidates is obtained through knowledge extraction, followed by manual annotation and automatic expansion. Experimental results on opening datasets show that the binary emotion commonsense library constructed in this paper improves the precision of sentiment analysis.
-
Sentiment Analysis and Social Computing2019, 33(6): 100-107.Short text sentiment analysis is a better method for judging the emotions of texts. It also has important applications in the fields of commodity reviews and public opinion monitor. The performance of the bidirectional recurrent neural network model based on the word attention mechanism relies heavily on the accuracy of word segmentation. In addition, the attention mechanism has more parameter dependencies, making the model less concerned with the internal sequence relationships of short texts. Aiming at the above problems, this paper proposes a Chinese short text sentiment analysis algorithm based on the character vector representation method combined with Self-attention and BiLSTM. Firstly, the short text is vectrized, then the BiLSTM network is used to extract texts context feature. Finally, the feature weights are dynamically adjusted by the self-attention mechanism, and the Softmax classifier obtains the emotion category. Experimental results on the COAE 2014 Weibo dataset and hotel review datasets show that character vectors are more suitable for short text than word-level text vector representations. The self-attention mechanism can reduce the external parameter dependence, so that the model can learn more key features of the text itself. Classification performance can be increased by 1.15% and 1.41%, respectively.
-
Sentiment Analysis and Social Computing2019, 33(6): 108-115.In aspect-based sentiment analysis, the attention mechanism is often combined in recurrent neural network to obtain the importance of different words. But it fails to obtain the importance of different sentences, nor the deep sentiment feature information. To deal with this issue, this paper proposes a double attention recurrent neural network. Double attention capture the importance of different words and different sentences in the word level and sentence level, respectively. Meanwhile, the aspects, the part of speech information and the position information are used as the auxiliary information of the model to identify the sentiment polarity of different aspects. Compared with IAN model on Restaurant dataset, Laptop dataset of the SemEval 2014, the classification accuracy is increased by 2.0% and5.2%, respectively. Compared with TD-LSTM model in Twitter dataset, the classification accuracy is raised by 1.7%.
-
Sentiment Analysis and Social Computing2019, 33(6): 116-123,140.In microblogging site, the comments under the original microblog reflect the original microblog’s content, users’ attitudes and certain related topics. To extract fine-grained information and affective meaning from those comments, we first propose to detect if a comment is targeted towards the microblog itself using three similarity methods. Then, a novel model is proposed for mining aspect-based opinion and affective meaning in microblogging comments. This model introduces emoticon sentiment and textual sentiment into LDA inference framework and achieves synchronized detection of aspect and affective meaning in comments. Experimental results demonstrate that the emoticon sentiment layer can improve the affective meaning recognition results.
-
Sentiment Analysis and Social Computing2019, 33(6): 124-131.In order to improve the sentiment classification of online reviews, a sentiment classification model based on semantic rules and deep learning model is proposed. We extend the semantic rule information based on sentiment dictionary, and embed it into common feature templates to form a more effective mixed feature template. And the Fisher discriminant criterion is applied to reduce the dimension of mixed feature templates to eliminate information redundancy. The deep learning model is based on a LSTM. The experiments on web crawled data show that the proposed method improves the classification for online reviews.
-
Sentiment Analysis and Social Computing2019, 33(6): 132-140.Online Social Networks (OSNs) rumor events detection has realistic significance to improve the quality of OSN information ecology environment and maintain social harmony. This paper, integrating the variation of rumor events features over time and the distribution of rumor events in time dimension, proposes a fuzzy algorithm based model to construct the dynamic time series features over time by introducing the idea of domain division. Meanwhile, we introduce the popularity, the ambiguity and the spread as three new features based on the communication theory of rumor events in sociology. The experimental results testify that our proposed model and new features improve the performance of automatic rumor events detection on Chinese Microblogs.
