2019 Volume 33 Issue 10
Published: 15 October 2019
-
Select all|
-
Survey2019, 33(10): 1-7.The medical knowledge graph is the cornerstone of intelligent medical applications. The existing medical knowledge graphs are not enough from the perspectives of scale, specification, taxonomy, formalization as well as the precise description of the knowledge to meet the needs of intelligent medical applications. We apply natural language processing and text mining techniques with a semi-automated approach to develop the Chinese Medical Knowledge Graph (CMeKG 1.0) . The construction of CMeKG 1.0 refers to the international medical coding systems such as ICD-10, ATC, and MeSH, as well as large-scale, multi-source heterogeneous clinical guidelines, medical standards, diagnostic protocols, and medical encyclopedia resources. CMeKG covers types such as diseases, drugs, and diagnosis/treatment technologies, with more than 1 million medical concept relationships. This paper presents the description system, key technologies, construction process and medical knowledge description of CMeKG 1.0, serving as a reference for the construction and application of knowledge graphs in the medical field.
-
Survey2019, 33(10): 10-17.As a key technique of information extraction, relation extraction is of great importance to many tasks such as automatic knowledge base construction and question answering systems. Distant supervision for relation extraction uses an external knowledge base as supervision signals to automatically label corpus, which can reduce the high cost of manual labelling. This paper presents a systematic survey to distantly supervised relation extraction. It classifies the existing methods into three types, including probabilistic graph-based, matrix completion-based and embedding-based ones. This paper also discusses the challenges and the future research directions of distantly supervised relation extraction.
-
Knowledge Representation and Acquisition2019, 33(10): 18-30.Distributed word embedding aims at using neural network framework to learn the low-dimension, compressed and dense representation vectors for words in corpus. This paper proposes a distributed word embedding based on multi-source information fusion (MSWE) . In the MSWE algorithm, the main improvements are focused on the following four aspects: (1) Through the explicit construction of context feature matrix, the co-occurrence of rare words and their context words can be retained in the language model, therefore, the structural semantic associations between words can be accurately reflected. (2) Through the descriptions and explanation texts of the words, the property semantic feature matrix of the words is constructed, which can effectively compensate the problem of the insufficient training due to the sparsity of the context. (3) The synonym and antonym matrix of the words are constructed, which makes the synonyms have a closer distance, and the antonyms have a farther distance in the word embedding space. (4) The multi-source feature matrices are integrated by the inductive matrix complement algorithm, and the various relationships of words are trained to get the low-dimensional embeddings. The experimental results show that the proposed MSWE algorithm shows an excellent performance on the six similarity evaluation datasets.
-
Machine Translation2019, 33(10): 31-37.Translation of a presenter's speech into other languages through speech recognition and machine translation in conference scenario is of great significance for cross-language communication. This paper proposes a domain-specific machine translation method based on external dictionary knowledge, so as to handle the translation of terminologies and professional expressions for a given conference. First, a constructed external dictionary is integrated through combining placeholder and concatenation fusion methods. The translation quality of domain related entity words and terminologies is significantly improved while maintaining coherence and fluency of the context. Second, classification-based domain adaptation can further improve the translation of the speech for a given conference while maintaining the overall quality of the general domain translation. Finally, an automatic domain adaptation training system is designed based on the above methods. The experimental results on Chinese to English translation task indicate that the proposed system achieves 9.22 BLEU improvement in average for sports, business and medical conference, without afftecting the general translation quality.
-
Machine Translation2019, 33(10): 38-46.Focused on Mongolian-Chinese machine translation, this paper proposes a Transformer-CRF algorithm to perform corpus preprocessing for Mongolian morphemes and Chinese word segmentation. Then the encoding-decoding model based on Tensor2Tensor is constructed. In order to learn more grammar and semantic knowledge from Mongolian corpus, this paper presents a morpheme quad-encoded word vector as the encoder input. In order to further alleviate the vocabulary limitation problem in neural network training, this paper introduces a proper noun dictionary into the translation model. Experimental results indicate that the model has improved translation quality in dealing with long-term dependence.
-
Information Extraction and Text Mining2019, 33(10): 47-56,72.Entity relation extraction identifies the relation between the target entity in the raw text, wichi is also widely used in text summarization, automatic question answering system, knowledge map, search engine, and machine translation. To deal with the complex structure and ambiguity in the Chinese sentences, this paper proposes a multi-feature self-attention entity relation extraction method. It employ a self-attention-based Bi-LSTM to capture the lexical, syntactic, semantic and position features. The experimental results on the Chinese COAE-2016 Task 3 and the English SemEval-2010 Task 8 show our method produces better performances.
-
Information Extraction and Text Mining2019, 33(10): 57-63.In the entity relationship extraction task, the distant supervision data set with substantial noise is often used. This paper applies ResNet to the distant supervision data set of relation extraction, to exploit its denoising ability by deepening the network. This paper also proposes a Gate module that can effectively improve the performance of deep residual networks, which can learn the importance between each feature channel. In addition, in order to further reduce the noise, this paper also proposes a new pooling layer called double pooling layer. The experimental results show that the proposed method achieves an improvement of 3% in precision and recall rate compared with the PCNN+ATT model.
-
Information Extraction and Text Mining2019, 33(10): 64-72.Relation extraction is a challenging task in information extraction, which is used to transform unstructured text into structured data. In recent years, deep learning models such as Convolutional Neural Network and Recurrent Neural Network have been widely used in relation extraction tasks and have achieved good results. To combine the advantages of CNN to extract local features and RNN to model in time series dependence, this paper proposes a convolutional recurrent neural network (CRNN) to extract phrase-level features and multi-granularity phrases for relation instances. The model is divided into three layers. Firstly, multi-granularity local features are extracted for the relation instance, and then the different granularity features are merged through the aggregation layer. Finally, the overall information of the feature sequence is extracted by RNN. In addition, this paper also explores the gains of various aggregation strategies for information fusion, and finds that the attention mechanism is the most prominent for the fusion of different granularity features. The experimental results show that CRNN is superior to state of the art CNN and RNN models with 86.52% of F1 scores on the SemEval 2010 Task 8 dataset.
-
Machine Reading Comprehension and Text Generation2019, 33(10): 73-80.Machine reading comprehension is a challenging task in natural language processing. Focused on fragment-extractive reading comprehension, this paper proposes an attention reading comprehension model based on multi-connect mechanism. The model more effectively exerts the role of attention mechanism in fragment extraction machine reading comprehension tasks through multiple connections. This model achieves an EM score of 71.175 and an F1 value of 88.090 in the final test data set of the Second Evaluation Workshop on Chinese Machine Reading Comprehension, CMRC 2018, ranking second.
-
Machine Reading Comprehension and Text Generation2019, 33(10): 81-89.In order to solve the opinion-problems of machine reading comprehension, an end-to-end deep learning model is proposed. In this paper, Bi-GRU is used to contextually encode passages and problems. And then four kinds of attentions, including the concatenated attention, the bilinear attention ,the element-wise dot attention and minus attention, are applied with the fusion of Query2Context and Context2Query attentions to obtain the comprehensive semantic information of the passage and the problem. This model further employs the multi-level attention transfer reasoning mechanism to obtain more accurate comprehensive semantics. The accuracy reaches 76.79% on the AIchallager 2018 opinion reading comprehension Chinese test data set. In addition, using the sentence sequence as input, the method could be boosted to an accuracy of 78.48%.
-
Machine Reading Comprehension and Text Generation2019, 33(10): 90-98.Sequence-to-sequence model based on encoder-decoder architecture is the mainstream of generative summarization method at present. However, the traditional encoder cannot effectively encode long document semantically, and ignores the hierarchical structure information of document. To deal with this issue, this paper propose to hierarchically encode the document: firstly, the word-level semantic representation is constructed, and then the sentence-level semantic representation is constructed from the word-level semantic representation. In addition, a semantic fusion unit is proposed to fuse the different levels of representation information as the final document-level representation. The experimental results show that the system performance is significantly improved according to ROUGE evaluation.
-
Question Answering and Dialogue System2019, 33(10): 99-108,118.Question paraphrase identification aims to identify whether two natural questions are semantic consistency. At present, paraphrase identification technology based on representation learning and deep neural network architecture has achieved good results. However, these methods often face bottlenecks with high complexity and difficulty in training. To tackle the problem, this paper proposes a fast method named multi-convolution self-interaction match (MCSM) model. This method combines multiple sentence features with word sense features to form a distributed representation. Then it utilizes convolution neural networks to capture phrase-level sentence representation. A self-interaction fusion technology is employed to obtain multi-granularity sentence vector representation, which can fully integrate word-level and phrase-level feature vector representation. Experimented on the Quora standard paraphrase identification corpus, the proposed method has comparable performance to the benchmark model based on bilateral multi-perspective match without external data. But has a training speed of its 19 times faster than the baseline, and the memory required is reduced by 80%.
-
Sentiment Analysis and Social Computing2019, 33(10): 109-118.This paper proposes a model using multi-channel convolution and bidirectional GRU for the sentiment analysis task. The multi-channel convolutional layer is used to extract the text feature with different granularity. The extracted features are merged into the bi-directional GRU with the attention mechanism to obtain the context sentiment features to classify the sentiment polarity. Attention mechanism adaptively perceives context information and extracts features that have strong influences on sentiment polarity using Maxout to solve the gradient vanishing in the training process. According to the classification accuracy on IMDb dataset and SST-2 dataset, the model performs better than neuron network models based on CNN-RNN architecture.
-
Sentiment Analysis and Social Computing2019, 33(10): 119-126.Sentiment classification towards Question-Answering reviews is a novel and challenging task in sentiment analysis community. However, due to the limited annotation corpus for QA sentiment classification, it is difficult to achieve significant improvement via supervised approaches. To overcome this problem, we propose a joint learning approach for QA sentiment classification, which treats QA sentiment classification as the main task while traditional review sentiment classification as the auxiliary task. In detail, we first encode QA review into a sentiment vector with main task model. Then, we propose an auxiliary task model to learn auxiliary QA sentiment information representation with the help of traditional review. Finally, we update the parameters both in main task model and auxiliary task model simultaneously through joint learning. Empirical results demonstrate the impressive effectiveness of the proposed joint learning approach in contrast to a number of state-of-the-art baselines.
-
NLP Application2019, 33(10): 127-134.Charge prediction is an important part in the field of intelligent judicature, which is aimed to predict the charge of the criminal subject based on the criminal facts. Criminal facts are the authentic and objective description of a case, in which the semantic importance of each word in criminal facts differs in the judgment of different charges. Existing studies ignore this semantic difference during modeling crime facts, and neglect the situation of cumulative punishment. In this paper, we incorporate the semantic differences of words into the attention mechanism in modeling crime facts. We then decompose the multi-label charges into several independent parts to realize the prediction under the condition of cumulative punishment. The experimental results show that the modeling based on semantic differences and multi-label transformation strategies are helpful to improve the effect of crime prediction, achieving F1 of 88.0% on CAIL2018 dataset.
-
NLP Application2019, 33(10): 135-142.Focused on language recognition on short utterances (with a duration less than or equal to 1s) and confusing speech, this paper investigates the performance of phoneme log likelihood ratio feature, the Mel frequency cepstral coefficient feature, and the deep bottleneck feature (DBF) , revealing that the DBF performs best in both tasks. To further improve recognition accuracy, the paper proposes an improved DBF-I-VECTOR system which, compared with the baseline of DBF-I-VECTOR on the Oriental Multilingual Recognition Competition Data, reduces the optimal equal error rate (EER) of short-term task from 12.26% to 10.55%, and the confusing task from 5.53% to 2.86% in respectively. It is also revealed that the Random Forest (RF) has the best classification performance in short-term task, and the Support Vector Machine (SVM) has the best classification performance in confusing task when compared with Cosine Distance Scoring (CDS) , Probabilistic Discriminant Analysis (PLDA) , Extreme Gradient Boosting (XGBoost) .
