Overlooked Video Classification in Weakly Supervised Video Anomaly Detection

Abstract

Current weakly supervised video anomaly detection algorithms mostly use multiple instance learning (MIL) or their varieties. Almost all recent approaches focus on how to select the correct snippets for training to improve performance. They overlook or do not realize the power of whole-video classification in improving the performance of anomaly detection, particularly on negative videos. In this paper, we study the power of whole-video classification supervision explicitly using a BERT or LSTM. With this BERT or LSTM, CNN features of all snippets of a video can be aggregated into a single feature which can be used for whole-video classification. This simple yet powerful whole- video classification supervision, combined with the MIL and RTFM framework, brings extraordinary performance improvement on all three major video anomaly detection datasets. Particularly it improves the mean average precision (mAP) on the XD-Violencefrom SOTA 78.84% to new 82.10%. These results demonstrate this video classification can be combined with other anomaly detection algorithms to achieve better performance. The code is pub-licly available at https://github.com/wjtan99/BERT_Anomaly_Video_Classification.