Regularized Multi-view Multi-metric Learning for Action Recognition

Abstract

Although multi-view datasets have become more accessible in the real-world applications, most state-of-the-art action recognition methods applied to those datasets rely on simple view agreement when combining local information from various views together. This leads to deteriorated performance in situations with view insufficiency and view disagreements. In this paper, we propose a novel framework for boosting action recognition performance by quantifying the connection between the viewpoint and an action. The proposed approach searches for the best combination of multiple views based on a co-learning strategy that simultaneously learns a local distance metric related to each action class and the relationships between each viewpoint and the action category. Consequently, the spatio-temporal representation of each action class in different viewpoints plays a key role in shaping the local distance metric space. We test our method on the IXMAS dataset and shows competitive performance compared to other state-of-the-art methods.