Action video retrieval based on atomic action vocabulary

Abstract

We propose an efficient action retrieval system that is based on a novel action representation and an effective video matching method. We represent actions with a hierarchical encoding scheme that at low-level measures local body parts motions, which then evolves into encoding of instantaneous global body motions and finally high-level description of actions through atomic action vocabulary. Atomic action vocabulary extends the notion of keyframe-based indexing techniques, where a long action video is decomposed into a sequence of atomic sub-actions matched from the vocabulary. Efficient video matching is achieved by exploiting precomputed inter-vocabulary distances so that global video distance between video sequences can be computed in a very efficient manner that is equivalent to index lookup operations with minimal additional computational loads. Combined with atomic action vocabulary, this can provide flexible video matching schemes of finding compound action sequences of arbitrary lengths. The proposed approach is evaluated on surveillance video and a public video dataset.