PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

Abstract

Today, scene graph generation(SGG) task is largely limited in realistic\nscenarios, mainly due to the extremely long-tailed bias of predicate annotation\ndistribution. Thus, tackling the class imbalance trouble of SGG is critical and\nchallenging. In this paper, we first discover that when predicate labels have\nstrong correlation with each other, prevalent re-balancing strategies(e.g.,\nre-sampling and re-weighting) will give rise to either over-fitting the tail\ndata(e.g., bench sitting on sidewalk rather than on), or still suffering the\nadverse effect from the original uneven distribution(e.g., aggregating varied\nparked on/standing on/sitting on into on). We argue the principal reason is\nthat re-balancing strategies are sensitive to the frequencies of predicates yet\nblind to their relatedness, which may play a more important role to promote the\nlearning of predicate features. Therefore, we propose a novel\nPredicate-Correlation Perception Learning(PCPL for short) scheme to adaptively\nseek out appropriate loss weights by directly perceiving and utilizing the\ncorrelation among predicate classes. Moreover, our PCPL framework is further\nequipped with a graph encoder module to better extract context features.\nExtensive experiments on the benchmark VG150 dataset show that the proposed\nPCPL performs markedly better on tail classes while well-preserving the\nperformance on head ones, which significantly outperforms previous\nstate-of-the-art methods.\n