Deep Kinship Verification and Retrieval Based on Fusion Siamese Neural Network

Abstract

Automatic kinship analysis, which aims to judge the kinship of different individuals, has been widely used in many real world applications such as helping missing persons reunite with their families and social media analysis. In this work, we focus on three practical and challenging tasks related to kinship analysis, i.e., kinship verification, tri-subject kinship verification and kinship retrieval. A deep fusion Siamese neural network is proposed to address these tasks in a flexible and progressive manner. Firstly, we propose a basic deep Siamese neural network for kinship verification to judge the kinship between individuals based on face images. More specifically, the Siamese neural network takes two input face images and then outputs the similarity between them. To improve the performance, a jury system is also introduced for multi-model fusion. Secondly, we integrate two basic deep Siamese neural networks for tri-subject kinship verification(father, mother and child), which is intended to decide whether a child is related to a pair of parents or not. Specifically, the kinship similairty score of the triplet for verification is obtained by weighting the similarity scores of the father-child and mother-child ones. Thirdly, the proposed deep Siamese neural network can be used to quantify the similarity between any two persons. Thus, it is natural and easy to extend its application to the kinship retrieval task by sorting the similarities between the candidates and faces in the database. We conduct experiments on the RFIW2021 dataset, and final results validate the effectiveness of our solution.