Video Semantic Concept Detection Based on Multi-modality Fusion

Abstract

Multiple kernel learning methods have a widespread application in visual concept learning and BoVW method has been widely used dues to its excellent categorization performance. However, most canonical multiple kernel learning methods employ a stationary kernel combination format which assigns a uniform kernel weights over the input space. And BoVW method aimed to resolve the problem that the time efficiency of BoVW method decreases as the visual data scales up. As it is true for human perception, learning from multi-modalities has become an effective scheme for various information retrieval problems. In this paper, we propose a novel multi-modality fusion approach for video search, where the search modalities are derived from a diverse set of knowledge sources. Our proposed approach, explores a large set of predefined semantic concepts for computing multi-modality fusion weights by a new method. Experimental results validate the effectiveness of our approach, which outperforms the existing multi-modality fusion methods.