2022 Volume 36 Issue 4
Published: 10 June 2022
-
Select all|
-
Survey2022, 36(4): 1-11.The language understanding processes in human brain is very complicated, involving multiple brain networks and processing mechanisms. Most previous work used strictly controlled experimental designs to investigate specific language phenomena. As a result, the research conclusions tend to be fragmented, hardly forming a picture about the brain language understanding. Recently, the emergence of deep learning has triggered technological changes in the field of language computation, and computational language models have reached or even surpassed human levels in multiple tasks. This brings the possibility of conducting global and highly ecologically valid language comprehension experiments, which will promote the develoyment of computational language methods in language cognition experiments. This article summarizes the related work of language cognition experiments using computational language methods, and anticipates the future development trends.
-
Language Analysis and Calculation2022, 36(4): 12-19.“不v1不v2” is one of the typical double negation structures with substantial complexity in Chinese. It includes three sub patterns: “不+助动词+不+v2” (不得不去), “不+是+不v2” (不是不好), and “不v1…不v2”. Taking “不v1不v2” as an example, with the conceptions of “non-truth-functional negation”, "factuality of verbs" and "negation focus", this paper formulates a strategy for automatically recognizing the double negation structure of “不v1不v2”. We also list out the auxiliary verb list and the non-factual verb, and further develop an automatic double negation recognition system. Tested with 28033 sentences, the proposed method achieves 97.89% recognition accuracy and 93.10% recall rate.
-
Language Analysis and Calculation2022, 36(4): 20-28.For pre-trained language models built on large-scale unsupervised corpus, such as BERT and XLNet, cross-entropy loss is routinely utilized as the loss function and models are typically evaluated by perplexity or other task losses. To deal with such mismatch between the training and evaluation loss functions, an improved pre-trained language model named RL-XLNet using Generative Adversarial Networks (GAN) and Reinforcement Learning (RL) is proposed. A generative model is trained to predict selected words, and a discriminative model is trained to predict whether the predicted token is correct or not. The reinforcement learning is adopted to train the generator. Through the interaction of the generator and the discriminator, the learning of semantic information is enhanced. Experiments on GLUE Benchmark and SQuAD question-answering Benchmark show that RL-XLNet outperforms traditional BERT and XLNet models in multiple natural language processing tasks: top-ranked in six tasks in GLUE, and top-ranked according to F1 scores in SQuAD task.
-
Language Analysis and Calculation2022, 36(4): 29-38.For Chinese Grammatical Error Correction (CGEC) task, although substitution errors account for the largest proportion of all the errors in the data set, no researcher has tried to incorporate phonological and visual similarity knowledge into the neural network-based GEC model. To tackle this problem, the article makes two attempts. First, this paper proposes a GEC model which incorporates with the confusion set knowledge based on the pointer network. Specifically, this model is Seq2Edit-based GEC model and use the pointer network to incorporate phonological and visual similarity knowledge. Second, during the training data pre-process stage, i.e., in the process of extracting edit sequences from wrong-correct sentence pairs, this paper proposes a confusion set guided edit distance algorithm to better extract substitution edit of phonological and visual similarity characters. The experimental results show that the two proposed methods can both improve the performance of the model and can provide complementary contributions; and the proposed model achieves the current state-of-the-art results in the NLPCC 2018 evaluation data set. Experimental analysis shows that compared with the baseline Seq2Edit GEC model, the overall performance gain of our proposed model is mostly contributed by correction of substitution errors.
-
Language Resources Construction2022, 36(4): 39-47,56.Discourse cohesion analysis plays a critical role in discourse understanding, and there exist differences in cohesion between English and Chinese. First, we explore proper strategies in annotating discourse cohesion, including clause, conjunction, reference and ellipsis. Then, we create 200 documents corpus which contains the information of cohesion alignment. Finally, this paper evaluates the corpus, discusses the problems and solutions in the annotation. The annotation consistency for clauses, connectives and reference in the corpus reaches 0.909, 0.876 and 0.920, respectively. The clause segmentation and connective recognition results show that the quality of tagged corpus meets the actual needs.
-
Language Resources Construction2022, 36(4): 48-56.We propose a methodology for constructing a city brain knowledge graph (CBKG) based on the resource description framework, IoT protocol and digital twin. Knowledge ontology and sub-ontology models for city brain are designed by coupling city elements and smart IoT standards. This knowledge graph model can integrate multi-source heterogeneous data, and therefore serving in the brain knowledge system for city-level intelligent operating system. We explore the event extraction under the city event ontology and designe a novel joint model to extract the event meta-theory. The results suggest that CBKG can support the intelligent management of the city in decision-makings. Future application of CBKG will couple with artificial intelligence, multi-sensor technologies, geographic information systems, and etc.
-
Information Extraction and Text Mining2022, 36(4): 57-65.The purpose of key phrase extraction is to extract a set of key phrases that can express the theme and contents of a document, which is of great significance for information retrieval and document classification. This paper proposes a semi supervised approach of Chinese key phrase extraction, on the basis of graph model and statistical features. The proposed approach applies the pre-training language model to represent phrases and articles, so as to reduce the dependence on a large number of annotated training data. Furthermore, a graph model is designed to represent the similarity space of candidate key phrases and iteratively calculate the importance of each phrase. Meanwhile, multiple statistical features are combined to further improve the accuracy of key phrase evaluation. The experimental results show that our proposed approach is more effective than the baselire methods for Chinese key phrase extraction.
-
Information Extraction and Text Mining2022, 36(4): 66-72.The 6th China Conference on Health Information Processing (CHIP2020) organized six evaluation tasks in Chinese medical information processing, among which task 1 was named entity recognition task of Chinese medical text. The main purpose of this task is to automatically identify medical named entities in medical texts. A total of 253 teams signed up for the evaluation, and 37 teams finally submitted 80 sets of results. The micro-average F1 is used as the final evaluation criteria, and the highest value of the submitted results reached 68.35%.
-
Information Extraction and Text Mining2022, 36(4): 73-80.Humor recognition is one of the emerging research issues in natural language processing. The special structure of dialogue makes humor recognition in dialogue more challenging in that, in addition to the current utterance, contextual information is also crucial for humor recognition. This proposes a BERT-based model for humor recognition in dialogue with enhanced context and semantic information. The model uses BERT to encode the initial utterance and speaker information, then employs sentence-level BiLSTM, CNN and Attention mechanism to enhance contextual information, and character-level BiLSTM and Attention mechanism to enhance semantic information. The experimental result shows that the model proposed can effectively improve the performance of humor recognition in dialogue.
-
Information Extraction and Text Mining2022, 36(4): 81-89.To balance the classification accuracy and computation cost of text classification model, this paper proposes a text classification model DistillBIGRU based on knowledge distillation. We construct the MPNetGCN model as the teacher model, select the bidectional gated recurent unit network as the student model, and obtain the final model DistillBIGRU through knowledge distillation. On multiple data sets, the average classification accuracy of the teacher model MPNetGCN is 1.3% higher than that of BERTGCN. And the DistillBIGRU achieves comparable classification effect to the BERT-Base mode with roughly 1/9 parameters of the latter.
-
Information Extraction and Text Mining2022, 36(4): 90-99.Chinese named entity recognition utilizes character embedding as the input of neural network models, which may give rise to the loss of certain semantic information since there is no clear word boundary in Chinese. To figure out the aforementioned issue, this paper proposes an entity recognition method based on multi-granular text representations. Firstly, the char and word representation are combined as the model input. Then the N-gram encoder is exploited to explore the potential word information in the N-gram which enriches the contextual representation of the sequence. The experimental results on the Weibo, Resume and OntoNotes4 dataset outperform the baseline and reach 72.41%, 96.52% and 82.83% respectively.
-
Information Extraction and Text Mining2022, 36(4): 100-110.In recent years, graph neural network model has been widely used in text classification tasks because of its ability to model non Euclidean data and capture global dependencies. In existing classification models based on graph convolution network, the composition method consumes too much memory and is difficult to adapt to new text. In addition, the existing methods of describing the global dependencies between graph nodes are not completely suitable for classification tasks. In order to solve the above problems, a probability distribution based graph convolution network for text classification is proposed. A heterogeneous relationship graph is constructed with words and labels in corpus as nodes. The global dependency relation between nodes is described by the probability distribution of words on each label, and the text representation learning is carried out by graph convolution. Experiments on 5 open text classification datasets show that the proposed model achieves better results wth a reduced graph size compared with other text classification network models.
-
The Key Technologies of Educational Cognition for Humanlike Intelligence2022, 36(4): 111-123.Automated essay scoring is a significant and challenging research topic, which has attracted the attention of scholars in the fields of artificial intelligence and education. Focuses on Chinese automated essay scoring, this paper proposes to exploit deep language analysis, including the application of spelling error corrector and grammar error corrector to analyze grammar level writing ability, the automatic rhetorical analysis and excellent expression recognition to reflect language expression ability, and the fine-grained quality analysis of essay to evaluate overall quality. We then propose an adaptive hybrid scoring model, combining linguistic features and deep neural networks. The experimental results on Chinese student essay datasets show that 1) incorporating deep language analysis features can effectively improve the performance of automated essay scoring; and 2) the grade and topic adaptive training strategy also improves the transferring and predication abilities.
-
The Key Technologies of Educational Cognition for Humanlike Intelligence2022, 36(4): 124-136.The challenge of creating an AI that can pass standard exams has attracted extensive research attention. In this paper, we focus on answering causal essay questions in high-school geography exams, which requires knowledge integration, multi-hop causal reasoning, and long-text answer generation. To this end, we define abstract event graph (AEG) to represent causal relations, and employ a pre-trained language model to construct an AEG from a corpus to integrate knowledge from multiple sources about high-school geography. Based on AEG, we employ graph neural network to unify structured and unstructured knowledge and realize multi-hop causal reasoning. On the GeoCEQA dataset with real high-school geography causal essay questions, our approach significantly outperforms the best baseline model by 0.8%—1.4% on ROUGE, by 0.4% according to BLEU, and by 4.2% according to human evaluation.
-
The Key Technologies of Educational Cognition for Humanlike Intelligence2022, 36(4): 137-145.The National College Entrance Examination is a standardized test that comprehensively evaluates the level of human knowledge and abilities, which serves as a more challenging question answering task. This paper designs an automatic question answering system for the history subject of National College Entrance Examination based on deep neural networks. One of the challenges for knowledge-enhanced question answering is the contextual sensitivity of knowledge: among the large amount of knowledge stored in the knowledge base, only a few pieces of knowledge are relevant to answering a certain question. In response to this challenge, this paper designs a knowledge-enhanced question answering system that combines knowledge retrieval and machine reading comprehension. Through the relevance ranking ability of the knowledge retrieval system and the knowledge positioning ability of the machine reading comprehension model, knowledge related to the question can be effectively discovered, thereby enhancing the performance of the question answering system. The experimental results show that the system can effectively answer questions in the history subject of National College Entrance Examination.
-
The Key Technologies of Educational Cognition for Humanlike Intelligence2022, 36(4): 146-155.Knowledge tracing (KT) is the task of assessing students’ evolving knowledge states in their learning process. Most existing methods have devoted significant efforts to explore different means to assess students’ knowledge state. However, the more fundamental issue of exercises representation receives relatively little attention. This paper proposes a Neural Knowledge Tracing Framework (NKTF) that integrates general exercises representation learning. Specifically, we first design a type of general exercises representation, which distinguishes exercises by their knowledge concepts, difficulty levels and other individual features. We then adapt several existing KT models to simultaneously capture the evolution of students’ knowledge state and learn the exercises representation. Finally, to predict students' future performance, we utilize the inner product of knowledge state and the representation of exercises to be answered to model the answering process. Extensive experimental results on three real-world datasets demonstrate that the NKTF can automatically learn precise and effective exercises representations. NKTF also effectively improves the performance of baseline KT models to outperform state-of-the-art methods.
-
The Key Technologies of Educational Cognition for Humanlike Intelligence2022, 36(4): 156-165.Essay generation is a challenging task in natural language generation. Unlike poetry and story generation, the essay generation demands accurate sentence-level semantics, clear argumentation structure and reasonable expression of core arguments. The state-of-art solution in this task,retrieval based method ignores the identification of logical argumentation relations, resulting in semantic incoherence and inversion of argumentation logic. In this paper, we apply proposes an argumentation identification method based on explicit semantic structure information, achieving better results than previous natural language inference models on the argumentation identification dataset. At the same time, the argumentation identification result is used as an explicit feature to apply to the sentence ordering model for essay generation, which effectively alleviates the logical inconsistency of the ordering model in the essay generation dataset and further improves the overall performance of the essay generation system.
-
The Key Technologies of Educational Cognition for Humanlike Intelligence2022, 36(4): 166-174.Machine Reading Comprehension (MRC) is a critical task in many real-world applications, which requires machines to understand a text passage and answer relevant questions. This paper studied the key technologies of textual semantic representation, candidate sentence extraction and language appreciation, and built the system for answering multiple choice questions and free-description questions. We have conducted some experiments on the Gaokao tests, finding that the system can achieve a certain degree of accuracy for both questions. In the future, we will explore to utilize more advanced techniques such as semantic representation, unified knowledge representation and aggregation, and transfer learning to improve the MRC system in complex reasoning, inductive analyzing and language appreciating.
