2021 Volume 35 Issue 2
Published: 10 March 2021
-
Select all|
-
Survey2021, 35(2): 1-18,32.Popularity prediction over online social networks plays an important role in various applications, e.g., recommendation, advertising, and information retrieval. Recently, the rapid development of deep learning and the availability of information diffusion data provide a solid foundation for deep learning based popularity prediction research. Existing surveys of popularity prediction mainly focus on traditional popularity prediction methods. To systematically summarize the deep learning based popularity prediction methods, this paper reviews existing popularity prediction methods based on deep learning, categorizes the recent deep learning based popularity prediction research into deep representation based and deep fusion based methods, and discusses the future researches.
-
Survey2021, 35(2): 19-32.Mental health problems are increasingly becoming one of the most serious and widespread public health issues in the world. The rise and popularity of social network brings a lot of data related to psychological state of its users. The research of applying social network data to automatically evaluate and detect users' mental health status has attracted more and more scholars in recent years. This paper reviews the relevant literature on the automatic assessment of mental health for social network users. Based on the existing literature, we sum up the concept and definition of automatic assessment of mental health, review the related researches at home and abroad from different aspects of assessment task, social network data-sets construction, the characteristics used in the assessment and so on. The characteristics of existing methods including feature engineering based methods and deep learning basedmethods are compared. Finally, we discuss the problems and challenges for this task, including assessment performance, data quality, privacy ethics, reason extraction and automatic intervention. Future research is suggested to combine other data streams and collaborate between patients, clinicians and data scientists to apply machine learning in causation extraction, prevention and counseling of mental health problems.
-
Language Resources Construction2021, 35(2): 33-40,51.It is of practical significance to develop a knowledge graph for the Beijing Winter Olympics 2022. To capture the complete glossary related to the Winter Olympics, this paper proposes a method of Chinese low frequency term expansion via bilingual iterative extension by exploiting the English corpus. Specifically, this paper uses a data set consisting of entries related to the Winter Olympics field in Wikipedia. This approach avoids the defects of existing Word2Vec approach which demanding large scale Chinese corpus with abundant target terms, which is not available. The experimental results show that compared with other set expansion methods, the proposed method has improved the quality of new extended words by more than 12%.
-
Language Resources Construction2021, 35(2): 41-51.There are a large number of explicit propositions in natural language, which contain the vital information about the text. Recognizing the proposition and analyzing the essential ingredients in propositions can reveal the logic behind the text and contribute to better natural language understanding. Based on the Baidupedia, we build an explicit proposition corpus and then propose two tasks: the automatic explicit proposition recognition and the essential explicit proposition ingredients analysis. Further, we construct a classification model based on BERT for the first task to determine whether a sentence is a proposition or not. And for the task two, we construct a sequence labeling model based on BERT-BiLSTM-CRF to select the essential ingredients in the propositions. The experimental results show that we achieve an accuracy of 74.95% (15.30% increase over the baseline) in task one, and a F-value of 90.74%(17.69% better than the baseline) in task two.
-
Machine Translation2021, 35(2): 52-60.The neural machine translation (NMT) model usually has a large amount of parameters, e.g. 100 million neurons in the transformer framework when the vocabulary is set to 30,000. To compress a complex and large-parameter NMT model, this paper proposes a progressive semi-distillation method. The semi-knowledge distillation is designed to obtain half weights from the teacher model as the starting point for the training of student models. The progressive semi-distillation method refers to, on the student model achieved above, apply the semi-knowledge distillation method again to obtain the full compression model. Experiments on two popular Chinese English and Japanese English datasets indicate, in terms of BLEU, at most 2.16 significant improvement to the baseline NMT model, 1.15 better than the word-level knowledge distillation method, and 0.28 higher than the sentence-level knowledge distillation method.
-
Information Extraction and Text Mining2021, 35(2): 61-68,77.High-quality paraphrase resources are of great help to improve the task of question answering system, machine translation and many other tasks. This paper is focused on paraphrase extraction of Chinese phrases, and proposes a sequence annotation model based on 2BiLSTM+CNN+CRF for phrase division in monolingual Chinese corpus. High-quality Chinese phrases are obtained through several filtering rules. After that, we adopt a method based on representation learning to obtain candidate paraphrase, in which Chinese phrase vector representation is learned through BattRAE model. In this paper, we extract candidate paraphrases based on the cosine similarity and filter them by rules. In the experiment, 500 phrasal paraphrases are randomly selected for manual evaluation, revealing an accuracy of 0.814 and a MRR of 0.826.
-
Information Extraction and Text Mining2021, 35(2): 69-77.Text classification is one of the core tasks in the field of natural language processing. To address the long text sequence, we propose the Hierarchical Networks with Mixed Attention (HMAN) model for text classification to capture the important parts of the text based on the hierarchical model. First, the sentences and documents are encoded according to the hierarchical structure of documents, and attention mechanism is applied at each level. Then, global target vectors, sentence specific target vectors are extracted by max-pooling to encode the document vectors. Finally, documents are classified according to the constructed document representation. Experimental results on the open datasets and industry text datasets show that the model has better classification performance, especially for long text with hierarchical structure.
-
Information Extraction and Text Mining2021, 35(2): 78-88.The hybrid text classification model based on convolutional neural network and recurrent neural network usually uses single-channel word embedding. Single-channel word embedding has low spatial dimension, leading that one-dimensional convolutional neural network fail to fully capture text features. This paper proposes a hybrid neural network text classification model combined with the channel features. The model uses two-channel word embedding to enrich text representation, fuses channel feature in the process of convolution, and optimizes the combination of spatial and temporal features. Tested on IMDB, 20NewsGroups, Fudan Chinese dataset and THUC dataset, the proposed model improves the classification accuracy by an average of 1% compared with the traditional methods, with a top increase of 1.3% on the THUC dataset.
-
Information Extraction and Text Mining2021, 35(2): 89-98.The neural topic models can effectively obtain the deep semantic features of the text, but the existing topic models are defected in negligence of the contextual information and the external knowledge. This paper proposes a topic model of judicial news based on neural autoregressive distribution estimator. The iDocNADEe is expanded with case elements as external knowledge, the attention mechanism is constructed by calculating the correlation between case elements and topic-relevant words to adjust weights of the hidden states in iDocNADEe. Then, the neural autoregressive algorithm is applied to calculate the weighted autoregressive conditional probability of the bidirectional hidden state of topic-related words. Experimental results show that compared with the baseline model, the perplexity is reduced by 0.66%, and the topic coherence is improved by 6.26% with the proposed method, as well as a significant higher document retrieval accuracy.
-
Question Answering and Dialogue System2021, 35(2): 99-106.Machine reading comprehension with unanswerable questions is a challenging task. In this paper, we propose a unified model, called U-Net, with three important components: answer pointer, no-answer pointer, and answer verifier. We introduce a universal node which processes the question and its context passage as a single contiguous sequence of tokens. The universal node encodes the fused information from both the question and passage, and plays an important role to predict whether the question is answerable and also greatly improves the conciseness of the U-Net. Different from the models based on pre-trained BERT, universal node fuses information from passage and question in a variety of ways and avoids the huge computation. The single U-Net model achieves the F1 score of 72.6 and the EM score of 69.3 on SQuAD 2.0, and the ensemble version, 74.9 and 71.4, respectively. Both version of U-Net models rank top among the models without a large scale pre-trained language model.
-
Question Answering and Dialogue System2021, 35(2): 107-115.The generative end-to-end dialog models suffer from generating monotonous and non-informative responses, which is a challenge in the dialog technology. As a highly structured knowledge source, knowledge graph can provide relevant information and topic transfer relationships that is essential to the continuation of the dialog. In this paper, a generative dialog model based on knowledge graph is proposed. First, a knowledge graph based mapping mechanism is used to process the dialog, then a copying mechanism is used to directly introduce the utterances contained in the knowledge graph into the generated responses and the information contained in knowledge graph is also used to guide the generation of responses by attention mechanism. On the "Knowledge-Driven Dialogue" dataset presented in the "2019 Language and Intelligent Competition", the proposed model outperforms the baseline model provided by competition organizer by 10.47% in character-level F1 and 4.6% in DISTNCT-1, respectively.
-
Sentiment Analysis and Social Computing2021, 35(2): 116-124,132.Aspect-based sentiment classification aims at judging the sentiment polarity of a particular aspect in a sentence. In the existing research on convolution-based neural networks, the maximum pooling operation is often used to extract text features as sentence representation in the pooling layer of the model. This operation does not consider the context divided by the aspect and fails to get finer-grained aspect context features. To solve this problem, this paper proposes a multi-feature piecewise convolution neural network (MP-CNN) model. According to the aspect, the sentence is divided into two parts of context, and in the pooling layer, the maximum pooling operation is used to extract the context features. In addition, this paper also integrates several auxiliary features into the model, such as relative position of words, part of speech and sentiment score of words in sentiment lexicon, and calculates the attention score of words through convolution operation. The experiments of SemEval 2014 and Twitter datasets confirm the best performance among the baselines.
-
Sentiment Analysis and Social Computing2021, 35(2): 125-132.This paper investigates the positive and negative emotion recognition of commodity reviews, and proposes a neural network model based on the Transformer model, which combines the multi-head self-attention layer and the convolution layer. The multi-head self-attention layer enriches the correlation between words, and the convolution operation is used for further feature extraction and fusion. Compared with Bidirectional Long Short-Term Memory Networks (Bi-LSTM), Bi-LSTM with attention and Text Convolutional Neural Networks (TEXTCNN), experiments show that the proposed model improves the top accuracy of emotion classification tasks by 4.12%, 1.47% and 1.36%, respectively.
-
Character Processing2021, 35(2): 133-140.A new offline handwritten Arabic word recognition algorithm is proposed to deal with its connected writing strokes and more similar words. The algorithm first establishes a structure model with fixed graphemes for each Arabic word category to be recognized, then segments the word samples into graphemes. Then a weighted Bayesian inference model is constructed from the grapheme features to word categories. The word recognition results are obtained by calculating the posterior probabilities of word categories. On the IFN/ENIT database, the proposed algorithm achieves as high as 90.03% accuracy.
