Incorporating Social Relationships for Multi-modality Person Re-identification

Abstract

Person re-identification (ReID) which aims to retrieve particular persons across non-overlapped cameras is very important to intelligent security system. Most previous methods of re-id focused on learning individual appearance feature embedding, but it is hard for the models to deal with many difficult situations such as occlusions. There are also some works those have attempted to integrate multi-modal cues, e.g. clothes and social relations. While showing improvements, they still have some limitations, such as designing too many objective functions to guide the model and relying on simple heuristics to combine multi-modal cues. In this paper, we propose a new framework to solve the above problems. Specifically, our model consists of three branches to extract and fuse information under three modalities respectively. The model can effectively utilize the information of person appearances, background scenes and social relationships in images to infer the identity of the person. The experimental results show that our method outperforms the state-of-the-art on the public datasets.