Generative and Attentive Fusion for Multi-spectral Vehicle Re-Identification

Abstract

Vehicle re-identification (Re-ID) task has far-reaching practical application value and research significance in busy traffic monitoring scenes. Most existing works are based on the single RGB modality or RGB-IR (Infrared) cross-modality. Still, it is difficult to be widely applied in multi-scene real life because of the inadequacy of a single modality at the limited daytime. Recently, multi-spectral Re-ID has been proposed to simultaneously use the RGB and IR spectral data to build a data cell that can complement each other to deal with this condition. For multi-spectral vehicle Re-ID, existing methods simply fuse features extracted from a data cell with multiple modalities, which ignores the large gap as well as the relationship between different modalities. To address this problem, in this work, we propose the Generative and Attentive Fusion Network (GAFNet) to fuse the multiple data sources for the vehicle Re-ID task. First, at the input-level, we enrich the data cell by adding the generated transitional modality images, which could involve different data distribution from the original modality images. Note that the newly generated images have high similarity with both the target modality and the source modality, therefore they would help reduce the gap between modalities and indirectly improve the consistency inside a data cell, and further enhance the weight of key information. Second, at the feature-level, we concatenate and fuse the features of the original data and the transitional modalities from different dimensions with the proposed attentive feature fusion module to further align the features and enhance their focused areas in the feature maps. Extensive experimental results on two standard benchmarks demonstrate that our method is effective and outperforms the state-of-the-art methods.