Abstract: In document-based dialogue systems, reference knowledge identification aims to identify the user query related information from the relevant document given the dialogue history. Based on ensemble self-knowledge distillation, this paper proposes a reference knowledge identification method. It uses the soft labels of reference knowledge provided by model itself and retains the model based on the proposed distillation loss. Experiments on the Doc2dial test set demonstrate that the F₁ measure and Exact Match rate reaches 76.71% and 63.92%, respectively, and the BLEU score of the reference knowledge powered by these reference knowledge is 1.7 higher than that by the RoBERTa model.