Counterfactual Sample Augmentation for Robust Dialogue Contradiction Detection

Dialogue contradiction detection aims to quantify the consistency of dialog systems. Current models based on human-written task-specific training data achieve poor performance when detecting dialogues generated by dialogue systems (i.e. poor robustness). To utilize the strategy of improving robustness by counterfactual samples, this paper proposes a counterfactual sample construction method for this task. This method first identifies the contradictory content in all contradictory samples, and then constructs corresponding counterfactual samples by deleting and adding contradictory content for contradictory and non-contradictory samples. Experiments on two mainstream detection models show that the proposed method can effectively improve the robustness of detection models in the real-world setting.