Text-based question generation is to generate related questions from a given sentence or paragraph. In recent years, sequence-to-sequence neural network models have been used to generate questions for sentences containing answers. However, these methods have the limitations: (1) the generated interrogatives do not match the answer type; and (2) the relevance of questions and the answer is not strong. This paper proposes a question generation model that based on answers and the contextual information. The model first determines interrogatives that match the answer type according to the relationship between the answer and the context information. Then, the model uses the answer and the context information to determine words related to questions, so that questions use words in the original text as much as possible. Finally, the model combines answer features, interrogatives, words related to questions with original sentences as inputs to generate a question. Experiments show that the proposed model is significantly better than the baseline systems.

The purpose of natural language understanding in task-oriented dialog system is to parse sentences entered by the user in natural language, extracting structured information for subsequent processing. This paper proposes an improved natural language understanding model, using BERT as encoder, while the decoder is built with LSTM and attention mechanism. Furthermore, this paper proposes two tuning techniques on this model: training method with fixed model parameters, and using case-sensitive version of pretrained model. Experiments show that the improved model and tuning techniques can bring 0.8833 and 0.9251 sentence level accuracy on ATIS and Snips datasets, respectively.

As a common linguistic phenomenon, ellipsis is common in texts, especially in short texts such as QA and dialogue. In order to understand the semantic information of short texts, we propose a multi-attention fusion model for Chinese ellipsis recovery. This model combines the context and the text information by gate mechanism, multi-attention and self-attention. Experiments on several short text corpora show that this model can efficiently detect ellipsis position and recover ellipsis content, facilitating better comprehension of short text.

Reading Comprehension (RC) refers to automatically answering questions on the given text, which has become a popular issue in natural language processing. Many deep learning RC methods have been proposed. However, these methods do not fully understand questions and the discourse, leading to poor performance of the model. In order to solve the problem, this paper proposes a reading comprehension method based on external knowledge and hierarchical discourse representation. The method uses the external knowledge and question types to enhance question comprehension. And the method utilizes the hierarchical discourse representation to improve the understanding of the discourse. Moreover, the two subtasks of the question type prediction and the answer prediction are jointly optimized in an unified framework. Experiments performed on the DuReader dataset show that the proposed method increased the performance by 8.2% at most.

Machine reading comprehension is a challenging task in natural language processing. Focused on fragment-extractive reading comprehension, this paper proposes an attention reading comprehension model based on multi-connect mechanism. The model more effectively exerts the role of attention mechanism in fragment extraction machine reading comprehension tasks through multiple connections. This model achieves an EM score of 71.175 and an F₁ value of 88.090 in the final test data set of the Second Evaluation Workshop on Chinese Machine Reading Comprehension, CMRC 2018, ranking second.

In order to solve the opinion-problems of machine reading comprehension, an end-to-end deep learning model is proposed. In this paper, Bi-GRU is used to contextually encode passages and problems. And then four kinds of attentions, including the concatenated attention, the bilinear attention ,the element-wise dot attention and minus attention, are applied with the fusion of Query2Context and Context2Query attentions to obtain the comprehensive semantic information of the passage and the problem. This model further employs the multi-level attention transfer reasoning mechanism to obtain more accurate comprehensive semantics. The accuracy reaches 76.79% on the AIchallager 2018 opinion reading comprehension Chinese test data set. In addition, using the sentence sequence as input, the method could be boosted to an accuracy of 78.48%.

Sequence-to-sequence model based on encoder-decoder architecture is the mainstream of generative summarization method at present. However, the traditional encoder cannot effectively encode long document semantically, and ignores the hierarchical structure information of document. To deal with this issue, this paper propose to hierarchically encode the document: firstly, the word-level semantic representation is constructed, and then the sentence-level semantic representation is constructed from the word-level semantic representation. In addition, a semantic fusion unit is proposed to fuse the different levels of representation information as the final document-level representation. The experimental results show that the system performance is significantly improved according to ROUGE evaluation.

Current image captioning is challenged the veracity of captions, i.e. an exact caption with tangible and specific entities is generated with a crude and monotonous captions ( e.g. “Messi takes the penalty kick” vs “a person is playing a ball.”). Focused on the identification and filling of person entities, this paper transform this task into a cloze issue with syntactic vacancy by removing the common person representation(e.g.“man”“player”) in the generated image caption. To introduce reading comprehension famework to address Who problem, this paper uses the R-Net to realize the acquisition and filling of the person name entity. In addition, we attempt to use the local and the global information to extract the person name entity, with local information indicating the source document that the image is located and the global information indicating the related documents from external links. Experiments show that the proposed method can effectively improve the quality of image caption generation and increase the BLEU by 2.93%.