2021 Volume 35 Issue 8
Published: 31 August 2021
-
Select all|
-
Survey2021, 35(8): 1-15.Adversarial attack and defense is a popular research issue in recent years. Attackers use small modifications to generate adversarial examples to cause prediction errors from the deep neural network. The generated adversarial examples can reveal the vulnerability of the neural network, which can be repaired to improve the security and robustness of the model. This paper gives a more detailed and comprehensive introduction to the current mainstream adversarial text example attack and defense methods, the data set together with the target neural network of the mainstream attack. We also compare the differences between different attack methods in this paper. Finally, the challenges of the adversarial text examples and the prospect of future research are summarized.
-
Survey2021, 35(8): 16-27.Neural machine translation has achieved good translation results on languages with abundant corpus, but it has poor performance on languages with scarce bilingual corpus resources such as Chinese-Vietnamese, this problem can be better alleviated by generating pseudo-parallel sentence pairs through word-level replacement of existing small-scale bilingual data. Considering the problem of multiple translations of one word in Chinese-Vietnamese word-level substitutions, so we studied the replacement based on larger granularity, and proposed the Chinese-Vietnamese pseudo-parallel sentence pair generation method based on phrase substitution. Use small-scale bilingual data for phrase extraction to construct a phrase alignment table, and expand it with entity phrases extracted from Wikipedia, after performing phrase recognition on bilingual data for Chinese and Vietnamese, use the phrase pair in the phrase alignment table that is more similar to the recognized phrase to replace, to achieve the phrase-level data enhancement, and train the final neural machine translation model together with the generated pseudo-parallel sentence pairs and the original data. Experimental results on Chinese-Vietnamese translation tasks show that pseudo-parallel sentence pairs generated by phrase substitution can effectively improve the performance of Chinese-Vietnamese neural machine translation.
-
Language Analysis and Calculation2021, 35(8): 28-37.Implicit discourse relation recognition is a challenging task in that it is difficult to obtain semantic informative and interaction-informative argument representations. The paper proposes an implicit discourse relation recognition method based on the Graph Convolutional Network (GCN). With the arguments encoded by fine-tuned BERT, the GCN is designed by concatenating the argument representations as feature matrix, and concatenating the attention score matrixes as adjacent matrix. It is hoped that the argument representations can be optimized by the self-attention and inter-attention information to improve implicit discourse relation recognition. Experimental results on the Penn Discourse Treebank (PDTB) show that the proposed method outperforms BERT in recognizing the four of implicit discourse relations, and it outperforms the state-of-the-art methods on Contingency and Expansion with 60.70% and 74.49% on F1 score, respectively.
-
Language Analysis and Calculation2021, 35(8): 38-46.Humor recognition is a challenge in the field of natural language processing. According to humor theories and cognitive linguistics, five types of distinctive aspects of humor are systematically analyzed, and a variety of humor features are derived. The experiment results show that the proposed features can better represent the latent semantic information of humor. Furthermore, the deep learning can benefit from these features for humor recognition.
-
Machine Translation2021, 35(8): 47-55.Pseudo-parallel sentences which may generated by word-level replacement of existing small-scale bilingual data are expected to alleviate the low resources language pairs Chinese-Vietnamese. Considering the multi-word translation in Chinese-Vietnamese, a phrase substitution based substitution method is proposed for Chinese-Vietnamese pseudo-parallel sentence pair generation method based on is proposed. A phrase alignment table is extracted from the small-scale bilingual data and then expanded by entity phrases collected from Wikipedia. Chinese and Vietnamese sentences in a bilingual corpus are then identified for the most similar phrases in the phrase table, and then replaced by such phrases. With such phrase-level expanded pseudo-parallel sentence pairs, experiment confirmed and improved performance for Chinese-Vietnamese neural machine translation.
-
Information Extraction and Text Mining2021, 35(8): 56-63.Aiming at the task of cross-domain few-shot relation classification, a Piecewise Attention Matching Network (PAMN) is proposed. To improve the sentence similarity algorithm for the task of relation extraction, two sentences are matched with their segmentations in PAMN, which can better estimate the similarity between relation classification instances. PAMN consists of encoding layer and sentence matching layer. At the encoding layer, PAMN uses the pre-trained model BERT to encode the sentence pair, divides the sentence into three segments according to the location of entity, and adapts the different domain through dynamic segmentation length. At the sentence matching layer, PAMN uses a text matching method based on the segmental attention mechanism to calculate the similarity between the query instance and each instance in the support set, and the average is taken as the similarity between the query instance and the support set. The experimental results show that PAMN has achieved the best results on the evaluation list in the field of FewRel 2.0 adaptation tasks.
-
Information Extraction and Text Mining2021, 35(8): 64-72,81.Keyphrase Generation (KG) is the task of capturing themes from a document, revealing the key information necessary to understand the content. Existing neural keyphrase generation approaches focus only on the token-level information while ignore sentence-level information such as document structure. In this paper, we incorporate the sentence-level inductive bias into KG and propose a new method named Sentence Selective Network (SenSeNet), which can automatically learn the sentence-level information and determine whether the sentence more likely to generate the keyphrase. We use straight-through estimator to train the model in an end-to-end manner and incorporate a weakly-supervised setting which is helpful for the training of the sentence selection module. Experiments show that our model successfully captures the document structure and reasonably distinguishes the significance of sentences, and consistent improvements achieved on two metrics in five datasets.
-
Information Extraction and Text Mining2021, 35(8): 73-81.Legal named entity recognition(LNER)is a fundamentaltask for the field of smart judiciary.This paper presents a new definition of LNER and a corpus of letters of proposal for prosecution named LegalCorpus. This paper proposes novel BERT based NER model for legal texts, named BERT-ON-LSTM-CRF (Bidirectional Encoder Representations from Transformers-Ordered Neuron-Long Short Term Memory Networks-Conditional Random Field). The proposed model utilizes BERT model to dynamically obtain the semantic vectors according to the context of words. Then the ONLSTM is adopted to extract the text features by modeling the input sequence and hierarchy. Finally, the text features are decoded by CRF to obtain the optimal tag sequence. Experiments show that the proposed model can achieve a F1-value of 86.09%, with 7.8% increased than the best baseline Lattice-LSTM.
-
Information Extraction and Text Mining2021, 35(8): 82-88,97.Entity relation extraction aims to extract semantic relations between entities from text. This task is well addressed for normative texts such as news reports and Wikipedia, but less touched for dialogue texts. Compared with standard text, dialogue is an interactive process, and such information hidden in the interaction challenges the entity relation extraction task. This paper proposes an entity relation extraction method that incorporates interactive information via cross-attention mechanisms. It also adopt the multitask learning to deal with the issue of unblance distribution. The experiments on DialogRE public dataset reveal a result of 54.1% F1 and 50.7% of F1c, which proves validity of the method.
-
Information Extraction and Text Mining2021, 35(8): 89-97.The present methods of entity relationship extraction are challenged by the noise of position vector and the lack of semantic representation. This paper proposed an entity relationship extraction model via both location based noise reduction and rich semantics. First, the model uses the position information and the word vector information trained by domain corpus to obtain the attention weight of each word.Then this weight is combined with the word vector trained by general field corpus to realize the noise reduction of position vector and the introduction of rich semantic information. Finally, the type of the entity relationship is determined by the weighted word vector. Evaluated on the i2B2 /VA corpus in 2010, experiments demonstrate a 76.47% F1 value, the best result on this corpus at present.
-
Sentiment Analysis and Social Computing2021, 35(8): 98-106.In aspect-based sentiment classification, the attention mechanism is often combined in recurrent neural network or convolutional neural network to obtain the importance of different words. However, such kind of methods fail to capture long-range syntactic relations that are obscure from the surface form, which would be beneficial to identify sentiment features directly related to the aspect target. In this paper, we propose a novel model named MemGCN to explicitly utilize the dependency relationship among words. Firstly, we employ the memory network to obtain the context-aware memory representation. After that, we apply graph convolutional network over the dependency tree to propagate sentiment features directly from the syntactic context of an aspect target. Finally, the attention mechanism is used to fuse memory and syntactic information. Experiment results on SemEval 2014 and Twitter datasets demonstrate our model outperforms baseline methods.
-
Sentiment Analysis and Social ComputingReinforcement Learning with User Long-term and Short-term Preference for Personalized Recommendation2021, 35(8): 107-116.Most of the existing recommendation methods based on deep reinforcement learning use recurrent neural network (RNN) to learn users' short-term preference, while ignoring their long-term preference. This paper proposes a deep reinforcement learning recommendation method combining both long-term and short-term user preference (LSRL). LSRL uses collaborative filtering to learn users' long-term preference representation and applies the gated recurrent unit (GRU) to learn user's short-term preference representation. The redesigned Q-network framework combines two types of representation and Deep Q-Network is used to predict users' feedback on items. Experimental results on MovieLens datasets show that the proposed method has a significant improvement according to NDCG and Hit Ratio compared to other baseline methods.
-
Sentiment Analysis and Social Computing2021, 35(8): 117-126.Existing information cascade prediction models are established by cascaded time series information or spatial topology. A social network-oriented information cascade prediction (Information Cascade Prediction, ICP) model is proposed to jointly model these two information based on deep learning. First, the Laplacian matrix is used to sample the cascaded nodes to generate a spatial sequence. Then the timing information and spatial structure information of the nodes are learned through the Bi-LSTM plus the graph convolutional network. And the information is finally cascaded through the attention mechanism. Experimental results show that the proposed model has higher prediction accuracy compared with existing method, reducing prediction error by about 1% to 8%.
-
Natural Language Understanding and Generation2021, 35(8): 127-134.The automatic question generation task aims to generate natural language questions for a paragraph of text. Existing neural question generation models mainly focused on using one sentence or the whole paragraph with target answer as the input. To better utilize the context of the target answer, this paper proposes a multi-input hierarchical attention sequence to sequence network to capture more valuable sentence information and richer semantic information of paragraph to generate high-quality questions. Experiments on SQuAD show that our method is better than the state-of-the-art in terms of BLEU4, and the response rate of this method is obviously better than the baseline system.
-
Natural Language Understanding and Generation2021, 35(8): 135-144.The sports news that summarized from text broadcast often fails to capture the background information. To address this issue, this paper proposes a method for automatic generation of NBA sports news. It designs a key event extraction algorithm to match the event points in the live text broadcast, and the first draft of news will be generated with the aid of the template with the key events highlighted. The final news will be automatically generated with the combination of the background information and important description, which are extracted from the constructed NBA sports domain knowledge graph. The constructed knowledge graph database has been released publicly, including 5893 entity nodes in 3 conceptual classes, 4 relationships and 27 attributes. Subjective and objective evaluation results on 50 randomly selected experimental results demonstrate the efficiency of the proposed method.
