Hierarchical contrastive learning for multi-label text classification

Abstract

Multi-label text classification presents a significant challenge within the field of text classification, particularly due to the hierarchical nature of labels, where labels are organized in a tree-like structure that captures parent-child and sibling relationships. This hierarchy reflects semantic dependencies among labels, with higher-level labels representing broader categories and lower-level labels capturing more specific distinctions. Traditional methods often fail to deeply understand and leverage this hierarchical structure, overlooking the subtle semantic differences and correlations that distinguish one label from another. To address this shortcoming, we introduce a novel method called Hierarchical Contrastive Learning for Multi-label Text Classification (HCL-MTC). Our approach leverages the contrastive knowledge embedded within label relationships by constructing a graph representation that explicitly models the hierarchical dependencies among labels. Specifically, we recast multi-label text classification as a multi-task learning problem, incorporating a hierarchical contrastive loss that is computed through a carefully designed sampling process. This unique loss function enables our model to effectively capture both the correlations and distinctions among labels, thereby enhancing the model's ability to learn the intricacies of the label hierarchy. Experimental results on widely-used datasets, such as RCV1-v2 and WoS, demonstrate that our proposed HCL-MTC model achieves substantial performance gains compared to baseline methods.