|
|
|
| Overview of 2018 NLP Challenge on Machine Reading Comprehension |
| LIU Kai, LIU Lu, LIU Jing, LV Yajuan, SHE Qiaoqiao, ZHANG Qian, SHI Yingchao |
| Baidu Inc., Beijing 100190, China |
|
|
|
|
Abstract Machine Reading Comprehension (MRC) is a challenging task in the field of Natural Language Processing (NLP) and Artificial Intelligence (AI). 2018 NLP Challenge on Machine Reading Comprehension (MRC2018) aims to advance MRC technologies and applications. The challenge releases the largest scale, open-domain, application-oriented Chinese MRC dataset, provides an open sourced baseline systems and adopts improved evaluation metrics. Over one thousand teams registered for this challenge and the overall performance of the participant systems have been greatly promoted. This paper presents an overall introduction to MRC2018, and gives a detailed description of the evaluation task settings, evaluation organization, evaluation results and corresponding result analysis.
|
|
Received: 27 June 2018
|
|
|
|
|
|
[1] Wei He, Kai Liu, Jing Liu,et al. DuReader: a Chinese machine reading comprehension dataset from real-world applications [C]//Proceedings of the Workshop on Machine Reading for Question Answering. Melbourne, Australia: Association for Computational Linguistics, 2018: 37-46.
[2] Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, et al. Bidirectional attention flow for machine comprehension[C]//Proceedings of ICLR,2017.
[3] Shuohang Wang,Jing Jiang. Machine comprehension using match-lstm and answer pointer[C]//Proceedings of ICLR, 2017.
[4] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries[C]// Text Summarization Branches Out: Proceedings of the ACL-04 Workshop. Barcelona, Spain: Association for Computational Linguistics, 2004: 74-81.
[5] Kishore Papineni, Salim Roukos, Todd Ward, et al. Bleu: A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics,2002:311-318.
[6] An Yang, Kai Liu, Jing Liu, et al. Adaptations of ROUGE and BLEU to better evaluate machine reading comprehension task[C]// Proceedings of the Workshop on Machine Reading for Question Answering. Melbourne, Australia: Association for Computational Linguistics, 2018: 98-104.
[7] Pennington Jeffrey, Richard Socher, Christopher Manning. Glove: Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).Association for Computational Linguistics,2014: 1532-1543. |
|
|
|
2018, Vol. 32 
