Window transformer for dialogue document: a joint framework for causal emotion entailment

The Causal Emotion Entailment (CEE) task aims to extract all potential pairs of emotions and corresponding causes from the unannotated emotion document in the conversational context. Most existing methods to solve CEE task follow a two-stage pipeline framework, in which the first stage is to identify emotional clauses and cause clauses and extract clause representation,separately. And in the second stage is to construct the final emotion and cause pairs. However, they ignore the effect of the distance between clauses on emotion-cause pair matching. Here, we construct a joint framework with Window Transformer to handle this problem. The pre-trained BERT and RoBERTa are used as the text encoder to generate a local representation of clauses in a given document. Meanwhile, we feed it into 2D Window Transformer to make the clause representation sensitive to the context within the Window and to obtain the dependencies between clauses. At the same time, the document ranks the candidate clauses to extract causal emotion entailments, which enhances the representation of clause pairs (emotion pairs and cause pairs) by kernel-based relative position embedding. Experimental results indicate that the framework acquires state-of-the-art results on the benchmark dataset.