Zero-Shot Generalizable End-to-End Task-Oriented Dialog System using Context Summarization and Domain Schema

Abstract

Task-oriented dialog systems empower users to accom-plish their goals by facilitating intuitive and expres-sive natural language interactions. State-of-the-art ap-proaches in task-oriented dialog systems formulate theproblem as a conditional sequence generation task andfine-tune pre-trained causal language models in the su-pervised setting. This requires labeled training datafor each new domain or task, and acquiring such datais prohibitively laborious and expensive, thus makingit a bottleneck for scaling systems to a wide rangeof domains. To overcome this challenge, we intro-duce a novel Zero-Shot generalizable end-to-end Task-oriented Dialog system, ZS-ToD, that leverages domainschemas to allow for robust generalization to unseen do-mains and exploits effective summarization of the dia-log history. We employ GPT-2 as a backbone model andintroduce a two-step training process where the goal ofthe first step is to learn the general structure of the dialogdata and the second step optimizes the response gen-eration as well as intermediate outputs, such as dialogstate and system actions. As opposed to state-of-the-artsystems that are trained to fulfill certain intents in thegiven domains and memorize task-specific conversa-tional patterns, ZS-ToD learns generic task-completionskills by comprehending domain semantics via domainschemas and generalizing to unseen domains seam-lessly. We conduct an extensive experimental evaluationon SGD and SGD-X datasets that span up to 20 uniquedomains and ZS-ToD outperforms state-of-the-art sys-tems on key metrics, with an improvement of +17% onjoint goal accuracy and +5 on inform. Additionally,we present a detailed ablation study to demonstrate theeffectiveness of the proposed components and trainingmechanism.