Injecting Commonsense Knowledge into Prompt Learning for Zero-Shot Text Classification

Abstract

The combination of pre-training and fine-tuning has become a default solution to Natural Language Processing (NLP) tasks. The emergence of prompt learning breaks such routine, especially in the scenarios of low data resources. Insufficient labelled data or even unseen classes are frequent problems in text classification, equipping Pre-trained Language Models (PLMs) with task-specific prompts helps get rid of the dilemma. However, general PLMs are barely provided with commonsense knowledge. In this work, we propose a KG-driven verbalizer that leverages commonsense Knowledge Graph (KG) to map label words with predefined classes. Specifically, we transform the mapping relationships into semantic relevance in the commonsense-injected embedding space. For zero-shot text classification task, experimental results exhibit the effectiveness of our KG-driven verbalizer on a Twitter dataset for natural disasters (i.e. HumAID) compared with other baselines.