SAFIT: Segmentation-Aware Scene Flow with Improved Transformer

Abstract

Scene flow prediction is a challenging task that aims at jointly estimating the 3D structure and 3D motion of dynamic scenes. The previous methods concentrate more on point-wise estimation instead of considering the correspondence between objects as well as lacking the sensation of high-level semantic knowledge. In this paper, we propose a concise yet effective method for scene flow prediction. The key idea is to extend the view of all points for computing point cloud features into object-level, thus simultaneously modeling the relationships of the object-level and point-level via an improved transformer. In addition, we introduce a novel unsupervised loss called segmentation-aware loss, which can model semanticaware details to help predict scene flow more accurately and robustly. Since this loss can be trained without any ground truth, it can be used in both supervised training and self-supervised training. Experiments on both supervised training and self-supervised training demonstrate the effectiveness of our method. On supervised training, 3.8%, 22.58%, 10.90% and 21.82 % accuracy boosts than FLOT [23] can be observed on FT3Ds, KITTIs, FT3Do and KITTIo datasets. On self-supervised scheme, 48.23% and 48.96% accuracy boost than PointPWC-Net [40] can be observed on KITTIo and KITTIs datasets.