Video Semantic Segmentation With Distortion-Aware Feature Correction

Abstract

Video semantic segmentation is active in recent years benefited from the\ngreat progress of image semantic segmentation. For such a task, the per-frame\nimage segmentation is generally unacceptable in practice due to high\ncomputation cost. To tackle this issue, many works use the flow-based feature\npropagation to reuse the features of previous frames. However, the optical flow\nestimation inevitably suffers inaccuracy and then causes the propagated\nfeatures distorted. In this paper, we propose distortion-aware feature\ncorrection to alleviate the issue, which improves video segmentation\nperformance by correcting distorted propagated features. To be specific, we\nfirstly propose to transfer distortion patterns from feature into image space\nand conduct effective distortion map prediction. Benefited from the guidance of\ndistortion maps, we proposed Feature Correction Module (FCM) to rectify\npropagated features in the distorted areas. Our proposed method can\nsignificantly boost the accuracy of video semantic segmentation at a low price.\nThe extensive experimental results on Cityscapes and CamVid show that our\nmethod outperforms the recent state-of-the-art methods.\n