2022 Volume 36 Issue 3
Published: 20 April 2022
-
Select all|
-
Language Analysis and Calculation2022, 36(3): 1-9.The discourse structure recognition task aims to identify the structure between adjacent discourse units for a hierarchical discourse structure tree .This paper proposes a pointer network model that integrates global and local information. It can effectively improve the ability of macro text structure recognition by considering the global semantic information and the closeness of the semantic relationship between paragraphs. The experimental results in the Chinese macro discourse Treebank(MCDTB) show that the proposed model outperforms the state-of-the-art model.
-
Language Analysis and Calculation2022, 36(3): 10-18.Humor plays an important role in daily communication. Existing works of humor level recognition tend to treat humor text as a whole, ignoring the inner semantic relations of it. Treating humor level recognition as a kind of natural language inference task, this paper divides humor text into two parts: "setup" and "punchline", and captures them with their mutual relations. A multi-granularity semantic interaction understanding network is proposes to capture semantic association and interaction in humor text from both word and clause granularity. We conduct experiments on public humor data set Reddit, and the accuracy of the model on this corpus is improved by 1.3% compared with the previous optimal results.
-
Language Analysis and Calculation2022, 36(3): 19-26.Event coreference resolution is a challenging task with wide application in event extraction, QA system, and reading comprehension. The best use the existing small-scale public corpus, this paper introduces a neural network model ECR_CDA based on cross-lingual data augmentation. This model enhances the corpus through the translation of Chinese and English corpus, and improves the performance of event coreference resolution via the cross-lingual learning of Chinese and English models by sharing the model parameters. The experimental results on ACE 2005 English test set show that ECR_CDA is superior to the most advanced baseline.
-
Ethnic Language Processing and Cross Language Processing2022, 36(3): 27-35.The training process of Mongolian acoustic model is a process where the model learns the relationship between pronunciation data and annotation data. Aiming at the modeling of Mongolianacoustic model based on phonemes, deal with the one-to-many mapping phenomenon between pronunciation and semantics, which will the decoding of Mongolian text will be wrong and will lead to the problem of low recognition rate of Mongolian speech recognition system. In this regard, this paper designs an End-to-End Mongolian acoustic model with both phonemes and letters used. Specifically, a Mongolian acoustic model based on BLSTM-CTC is described, and a momentum training algorithm is applied. The experimental results show that the proposed method can effectively reduce the word error rate of heteromorphous homophones in Mongolian speech recognition system.
-
Ethnic Language Processing and Cross Language Processing2022, 36(3): 36-44.Sentiment classification of Vietnamese online comments is the viald for the opinion analysis of Vietnamese event . As Vietnamese is a low-resource language, the cross-lingual sentiment classification can be performed with the help of Chinese annotated corpus to help the sentiment polarity prediction of Vietnamese. In this paper, a cross-lingual sentiment classification model of Chinese and Vietnamese comments incorporating confrontational topic features is proposed. The topic distributions of Chinese and Vietnamese are introduced into the model as external knowledge, and a gate layer is used to encode representation from topic representations with semantic representations. The model is optimized to learn the representations with the smallest differences in language distributions through the adversarial learning to finally complete the sentiment classification task. The experimental results show that the proposed model can significantly improve marco F1 values compared with several baseline models.
-
Ethnic Language Processing and Cross Language Processing2022, 36(3): 45-53,63.Recently, the evaluation method of text generation based on pre-trained language model has gained attention, which evaluates the quality of generated text by computing the granularity similarity of sub-words of two sentences. However, for languages that contain many adhesive morphemes, such as Vietnamese and Thai, a single syllable or sub-word cannot form the semantic integrity, which means that the sub-word granularity matching method cannot fully represent the semantic relationship between two sentences. Therefore, we propose a text generation evaluation method with multi-granularity features of sub-words, syllables, and phrases. After the representation of text is obtained by MBERT, the semantic similarity of syllables and phrases is introduced to enhance the evaluation model of sub-words. Experimental results on such tasks as cross-language summarization, machine translation, and data screening show that, compared with ROUGE, BLEU based on statistical evaluation and Bertscore based on deep semantic matching, the proposed metric correlates better with human judgments.
-
Information Extraction and Text Mining2022, 36(3): 54-63.This paper proposes, a method of identifying named entities based on neural network with domain knowledge to identify 12 types of process entities including parts, engineering drawings, reference standards and attributes. According to the characteristics of process entities, this method uses domain dictionaries and rules to pre-identify candidate entities to form pre-recognition features, which are then fed to the CNN-BiLSTM-CRF neural network model. The experimental results show that, by adding pre-recognition entity features, the F1 value is increased from 90.99% to 93.03%.
-
Information Extraction and Text Mining2022, 36(3): 64-72.Named Entity Recognition (NER) has been receiving much attention as a basic work in the field of Natural Language Processing. Due to the complexity of the structure of Chinese words, especially for the combined entities, there are still two issues should be addressed: the associated information between words and characters, and the varied sequence length. To deal with the first issue, a highway network that integrates bi-direction attention is proposed to extract the combination of effective characters in a word for a more adequate representation. For the second issue, this paper provides a highway network combined with self-attention, to obtain relevant features of context from multiple perspectives and levels. Experiments on the OntoNotes V4.0 corpus show the effectiveness of the proposed models: the best performance without using a large pre-trained language model.
-
Information Extraction and Text Mining2022, 36(3): 73-81,90.In recent years, neural network models trained on a large number of labeled samples boost the performance of named entity recognition. However, collecting enough labeled samples in various domains is very expensive, which reveals the importance of rapid domain transferring. Given only unlabeled data of a target domain, this paper attempts to automatically construct corpora with partial annotation in the target domain and model it. Firstly, we use two different methods to annotate unlabeled data automatically. Then, we keep consistent annotations while removing those with different annotations, which reduces erroneous annotations as much as possible and generates a partial-annotated corpus. Finally, we propose a new entity recognition model based on partial annotation learning. Experiment results of transferring from the news domain to the social-media domain as well as finance domain prove that the proposed approach effectively improves the domain adaptation performance of named entity recognition at a low cost. With the addition of pre-trained language model BERT, this method still exhibits good performance.
-
Information Extraction and Text Mining2022, 36(3): 82-90.Quote attribution in novels aims at determining who says a quote in a given novel. This task is important for assigning appropriate voices to the given quotes when producing vocal novels. In order to fully express the difference of quote types and the semantic features in the context, this paper proposes a Rule-BertAtten method for quote attribution in Chinese novels. The quotes are divided into four categories: the quote with explicit speaker, the quote with pronoun speaker with one-match gender, the quote with pronoun speaker with multi-match gender and the quote with implicit speaker. According to these categories, a rule-based method and the BERT word embedding methods with Attention are applied respectively. The experiment result shows that our method is more accurate than previous approaches.
-
Question Answering and Dialogue System2022, 36(3): 91-100.Automatically question generation (QG for short) is to automatically generate the corresponding interrogative sentence of the target answer under the given context. . In this paper, we take advantage of pre-trained language model and apply the UNILM on encoder-decoder framework of question generation. In particular, in order to solve the problems of "exposure bias" and "mask heterogeneity" in the decoding phase of model, we examine the noise-aware training method and transfer learning on UNILM to raise its adaptability Experiments on SQuAD show that our best model yields state-of-the-art performance in answer-aware QG task with up to 20.31% and 21.95% BLEU score for split1 and split2, respectively, and in answer-agnostic QG task with 17.90% BLEU score for split1.
-
Question Answering and Dialogue System2022, 36(3): 101-108.Sequence to sequence generation model mostly uses the way of adding external emotional words when the model transfers the emotional state, which is defected in generation responses with emotional word stacks and lack of emotional information context. To address this issue, this paper proposes an emotion controllable conversation generation model based on position awareness. In the encoding process, the current input word vector and position vector jointly participate in encoding. Without affecting the current input, the preceding context is encoded by an additional leyer. In the decoding process, the masked model is used to force the model to understand and learn the content. The joint training of the encoder and the decoder could generate a grammatical emotion response. The experimental results show that the position awareness further characterizes the potential structural information on the data and improves the model quality.
-
Information Retrieval and Question Answering2022, 36(3): 109-119.In order to improve the ability of semantic retrieval in search engines, this paper proposes a semantic relevancy model for news entity and text. A corpus 10, 000 financial news with the semantic relatedness between entities in headlines and text has been manually annotated. Then the BERTCA (Bidirectional Encoder Representation from Transformers Co-Attention semantic relevancy computing) model has been established using this corpus. Through the co-attention mechanism, this model can obtain the semantic matching between the entity and text, and it can not only calculate the degree of correlation between entity and text, but also determine the degree of correlation according to the semantic relevancy. The experimental results show that the accuracy of the proposed model surpasses 95%, which is better than the state-of-the-art models.
-
Information Retrieval and Question Answering2022, 36(3): 120-127.To further improve current recommendation algorithms, such as Matrix Factorization, a method of Deep Attention Matrix Factorization (DeepAMF) are introduced in this paper. First, the multi-layer perceptron technology is applied to obtain a better feature representation and got the relational information through the dot product operation during the original input, which are named as Deep Matrix Factorization (DeepMF). Then multi-layer attention network is exploited to to obtain the user's preference for the item. Besides, the dot product operation is applied before the output to obtain the related information of the feature expression. And the module was called. Experiments on four public data sets prove the effectiveness of the MAMF algorithm.
-
Sentiment Analysis and Social Computing2022, 36(3): 128-135.A large number of tourists record their mood on social media, with abundant tourists' emotional information about the scenic spot in their travel notes. To provide better scenic spot information, this paper proposes a emotional summary method for multiple scenic spot travel notes. It combines the pointer generation network and the maximum boundary correlation algorithm to build an end-to-end neural network summary generation model. The proposed model attaches importance to the emotional information while abstracting the text. The experimental results show that the proposed model is effective on a self-built data set.
-
Sentiment Analysis and Social Computing2022, 36(3): 136-145.Aspect terms extraction is the task of extracting the entity attributes commented in the opinions, which is a sub-task of aspect-based sentiment analysis. To improve the current solution base on convolutional neural networks (CNN), this paper proposes an aspect terms extraction model based on double embedding and multiple attentions. It better captures the long-range dependencies by a joint Non-local networks, betters capture the characters in the text by the spatial attention combined with jump connections. Experimented on the laptop dataset and the restaurant dataset, the proposed method achieves F1 values of 83.39% and 76.26%, respectively, better than multiple baseline models.
-
Sentiment Analysis and Social Computing2022, 36(3): 146-153.Aiming at the problem that attention mechanism and convolutional neural network model cannot explore the dependencies between long-distance words and related syntactic constraints in a sentence in aspect-level sentiment analysis research, and the context words unrelated to grammar are used as aspect sentiment judgment clues. This paper proposes an aspect-level emotion classification model (ASGCN-AOA) combining graph convolution network (GCN) and attention-over-attention (AOA) neural network. Firstly, a bidirectional long short-term memory network is used to model the aspect-specific representations between context words. Then, in the dependency tree of each sentence, the corresponding graph convolution network (GCN) is established to obtain the aspect feature of considering syntactic dependence and long-distance multi-word relationship simultaneously. Finally, the AOA attention mechanism captures the interaction and representation between aspect words and context sentences, and automatically pays attention to important parts of sentences. Experiments were carried out on five data sets: Twitter, Lap14, Rest14, Rest15 and Rest16. Accuracy and Macro-F1 indicators were used to evaluate. Experimental results show that the model presented in this paper is significantly improved compared with other related aspect-based analysis algorithms.
-
Sentiment Analysis and Social Computing2022, 36(3): 154-161.In recent years, classification and rating of social emergency event have attracted more and more attentions. Most of the current studies adopt the rule-based methods to identify the evidences for event judgement. Inspired by the idea of event extraction, this paper proposes the event judgement method via BiLSTM (Bi-directional LongShort-Term Memory) and CRF (Conditional Random Fields) based on event classification and evidence extraction. The social emergency event classification is performed, and then the event evidences are extracted based on event type. In the end, the rating of social emergency event is determined by the attention mechanism with event type and evidences. Experimental results show that the proposed method is more robust than rule-based ones, and effective in the social emergency event judgment.
-
Sentiment Analysis and Social Computing2022, 36(3): 162-172.The second instance judgment prediction task aims to predict the judgment results of appeal trials based on the first instance judgment, newly discovered facts, and appeal reasons. There are two challenges to solve the second instance judgment prediction task. One is how to capture the cognitive similarities and differences between superior and lower courts on case facts. The other is how to make the prediction interpretable. To address these challenges, we propose SIJP-SML, which is a second instance decision prediction framework based on sequential multi-task learning. SIJP-SML models the complete trial logic from the first instance to the second instance through two time-dependent multi-task learning components. Cognitive representations of superior and lower courts are extracted from case facts and integrated to produce the second trial decision. To improve prediction interpretability, SIJP-SML takes court-view-generation as one of the learning tasks to output judgment rationales with readability. Experimental results on a dataset of more than 60K second instance judgment documents reveal that SIJP-SML outperforms all the baseline methods.
