Feature Re-Balancing for Long-Tailed Visual Recognition

Abstract

Despite the recent success of visual recognition on artificially balanced datasets, the performance degrades heavily in face of the long-tailed distribution. Existing methods typically tackle this problem by re-balancing the distribution in the data space. However, we observe that more balanced data distribution can not effectively alleviate the problem of uneven feature distribution, still leading to a heavily biased classifier. In this paper, we propose a novel re-balancing framework, Feature Re-Balancing (FeatRB), which directly re-balances the distribution in the feature space by combining the long-tailed initial features and the generated virtual features. The key ideas of FeatRB include: 1) Generating the virtual features. First, we calculate the class-wise feature mean and variance based on the past learned representations and store them in a memory bank. Then we generate virtual features based on the memory bank. And to increase the diversity of generated features, we transfer the variance from similar classes to tail classes. 2) Utilizing the generated features. We introduce a simple but effective sampling strategy, Effective Number Reversed Sampling (ENRS), to assign larger sampling probability for tail classes. 3) Updating the memory bank. We propose an updating method, Adaptive Updating (AU), which adaptively updates the memory bank in the training process to further improve the diversity. By increasing the intra-class diversity, FeatRB enlarges the spatial span for the features of tail classes. Therefore, the discriminative power of tail classes can be enhanced, and then the biased classifier can be calibrated. Extensive experiments on three widely used large-scale long-tailed datasets show that our FeatRB surpasses the current state-of-the-art methods.