The Third Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC 2019)

This is the official repository for CMRC 2019.

More details:


April 8, 2020 Add dataset description paper
August 1, 2019 Qualifying data has been released, check data directory. (Note that, json key 'answers' no longer contains answers.)
June 10, 2019 Development data has been released, check data directory.
May 24, 2019 Baseline system has been released, check baseline directory.
May 23, 2019 Train/Trial data have been released, check data directory.
May 17, 2019 We are expected to release task introduction and call for participation on late May or early June. Stay tuned!

Directory Guide

Baseline System

We provide a BERT-based baseline system for participants (check baseline directory for more info).

Results on other sets will be annouced later.

QAC: Question-Level Accuracy

PAC: Passage-Level Accuracy

Data Passage # Query # QAC PAC Fake Candidates Availability
Trial Data 139 1,504 71.941% 28.776% No Public
Train Data 9,638 100,009 N/A N/A No Public
Development Data 300 3,053 70.586% 13.333% Yes Public
Qualifying Data 500 5,081 70.01% 8.20% Yes Semi-Hidden
Test Data - - - - Yes Hidden


If you wish to use our data in your research, please cite our paper:

  title={A Sentence Cloze Dataset for Chinese Machine Reading Comprehension},
  author={Cui, Yiming and Liu, Ting and Yang, Ziqing and Chen, Zhipeng and Ma, Wentao and Che, Wanxiang and Wang, Shijin and Hu, Guoping},
  journal={arXiv preprint arXiv:2004.03116},

Organization Committee

Host: Chinese Information Processing Society of China (CIPS)
Organizer: Joint Laboratory of HIT and iFLYTEK (HFL)
Sponsor: iFLYTEK Co., Ltd. and iFLYTEK Research (Hebei)

Evaluation Co-Chairs

Ting Liu, Harbin Institute of Technology
Yiming Cui, iFLYTEK Research

Official Discussion QQ Group

You can join our QQ group for discussion purpose.


Official HFL WeChat Account

Follow Joint Laboratory of HIT and iFLYTEK Research (HFL) on WeChat.


Contact us

Any problems? Feel free to concat us.
Email: cmrc2019 [aT] 126 [DoT] com
Forum: CodaLab Competition Forum