Efficient Human Pose Estimation via Multi-Head Knowledge Distillation

Abstract

Nowadays, neural networks are becoming increasingly swelling to achieve high accuracy and better adaptability. For the task of human pose estimation, heavy neural networks are applied to achieve higher performance. But more extensive networks would lead to lower inference speed and consume more computing resources. How to acquire preferable accuracy even with a smaller model has become a valuable research subject. In this paper, we applied Knowledge Distillation to improve the performance with lightweight model. But still, the performance disparities between teacher networks and student networks exist. To further narrow the gap, we proposed the Multi-Headed Architecture to promote the accuracy of smaller models, raising performances to a similar level to larger models.