Clip-based similarity measure for query-dependent clip retrieval and video summarization

Abstract

This paper proposes a new approach and algorithm for the similarity measure of video clips. The similarity is mainly based on two bipartite graph matching algorithms: maximum matching (MM) and optimal matching (OM). MM is able to rapidly filter irrelevant video clips, while OM is capable of ranking the similarity of clips according to visual and granularity factors. We apply the similarity measure for two tasks: retrieval and summarization. In video retrieval, a hierarchical retrieval framework is constructed based on MM and OM. The validity of the framework is theoretically proved and empirically verified on a video database of 21 h. A query-dependent clip segmentation algorithm is also proposed to automatically locate the potential boundaries of clips in videos. In video summarization, a graph-based clustering algorithm, incorporated with the proposed similarity measure, is adopted to detect the highlighted events reported by different newscasts.