Empathetic Dialogue Generation by Incorporating Commonsense Knowledge Based on Multi-Head Attention Mechanism

Abstract

Significant progress has been made in dialogue generation techniques. However, to meet the demands of human communication more effectively, researchers have incorporated empathy into dialogue generation. Empathy, a crucial component of interpersonal communication, aids in enhanced understanding of others' emotions and feelings. The latest methods realize this enhanced understanding by introducing commonsense knowledge through the use of unified encoding for non-emotional knowledge and employing a simple vector concatenation for integrating commonsense knowledge. However, this approach may potentially decrease the influence of certain commonsense knowledge features and also lacks clear representations of the interrelations among various knowledge components. An empathetic dialogue model ATT-EDM is proposed to address these limitations, which leverages the multi-head attention mechanism to effectively fuse commonsense knowledge. The model individually encodes the five introduced relationships-xReact, xWant, xNeed, xIntent, and xEffect, preserving the distinctive features of each commonsense knowledge. It leverages a multi-head attention mechanism to integrate knowledge, computes each type of knowledge separately in the attention layer to reflect their respective influences more accurately, and effectively captures the interconnections among various commonsense knowledge components. Experimental results obtained using the EmpatheticDialogues dataset demonstrate that the proposed model outperforms baseline models in terms of the Perplexity (PPL), accuracy, and Distinct-2 metrics. Specifically, it achieves a reduced PPL of 36.435 0, with the accuracy and Distinct-2 metrics reaching 37.96% and 3.345 5, respectively. This enables the generation of high quality responses that are content-rich and empathetic.