Zero-Shot Learning for Skeleton-based Classroom Action Recognition

Abstract

Classroom behaviours reflect teaching efficiency and performances significantly. Different from daily actions such as running and walking, most classroom behaviours are in sitting positions (e.g. listening, reading and playing mobile phones), which makes the action features of those behaviours relatively close along with occlusion problems. Recently, the graph convolution operation on human skeleton data can effectively extract the skeleton point information, which provides the possibility for the effective recognition of classroom action. Inspired by this, in our research paper, a kind of class action recognition method on the basis of graph convolution was proposed, which takes classroom video as input and recognizes student actions through a skeleton extraction module, followed by a feature extraction module based on graph convolution and a zero-shot feature classification module. Comprehensive experiments show that our model has achieved significant performance in classroom behaviour recognition tasks with an accuracy of 60.67% in collected classroom dataset. Simultaneously, the validity of the model is verified in the ablation experiment.