Hidden-Concept Driven Multilabel Image Annotation and Label Ranking

Abstract

Conventional semisupervised image annotation algorithms usually propagate labels predominantly via holistic similarities over image representations and do not fully consider the label locality, inter-label similarity, and intra-label diversity among multilabel images. Taking these problems into consideration, we present the hidden-concept driven image annotation and label ranking algorithm (HDIALR), which conducts label propagation based on the similarity over a visually semantically consistent hidden-concepts space. The proposed method has the following characteristics: 1) each holistic image representation is implicitly decomposed into label representations to reveal label locality: the decomposition is guided by the so-called hidden concepts, characterizing image regions and reconstructing both visual and nonvisual labels of the entire image; 2) each label is represented by a linear combination of hidden concepts, while the similar linear coefficients reveal the inter-label similarity; 3) each hidden concept is expressed as a respective subspace, and different expressions of the same label over the subspace then induce the intra-label diversity; and 4) the sparse coding-based graph is proposed to enforce the collective consistency between image labels and image representations, such that it naturally avoids the dilemma of possible inconsistency between the pairwise label similarity and image representation similarity in multilabel scenario. These properties are finally embedded in a regularized nonnegative data factorization formulation, which decomposes images representations into label representations over both labeled and unlabeled data for label propagation and ranking. The objective function is iteratively optimized by a convergence provable updating procedure. Extensive experiments on three benchmark image datasets well validate the effectiveness of our proposed solution to semisupervised multilabel image annotation and label ranking problem.