Spatial-temporal histograms of gradients and HOD-VLAD encoding for human action recognition

Abstract

Automatic human action recognition is a core functionality of systems for video surveillance and human object interaction. In the whole recognition system, feature description and encoding represent two crucial key steps. In order to construct a powerful action recognition framework it is important that the two steps must provide reliable performance. In this paper, we proposed a new human action feature descriptor which is called spatial-temporal histograms of gradients (SPHOG). SPHOG is based on the spatial and temporal derivation signal, which extracts the gradient changes between consecutive frames. Compare to the traditional descriptors histograms of optical flow, our proposed SPHOG costs less computation resource. Vector of Locally Aggregated Descriptors (VLAD), which is a popular encoding approach for Bag-of-Feature representation. There is a main drawback of VLAD that it only considers the difference between local descriptor and their centroids. In order to resolve the weakness, we proposed a improved VLAD method called HOD-VLAD, which complementary the distribution information of local descriptors by computing a weight histograms of distance. We validated our proposed algorithm for human action recognition on three public available datasets KTH, UCF Sports and HMDB51. The evaluation experiment results indicate that the proposed descriptor and encoding method can improve the efficiency of human action recognition and the recognition accuracy.