Attention Guided Unsupervised learning of Monocular Visual-inertial Odometry

Abstract

Visual-inertial Odometry (VIO) provides cars with position information by fusing data from a camera and inertial measurement unit (IMU) which are both widely equipped on intelligent vehicles. Recently, unsupervised VIO has made great progress. However, existing VIOs mainly concatenate features extracted from different domains (visual and inertial), leading to inconsistency during integration. These methods are also difficult to scale to longer sequences because absolute velocity is not available. Hence, we propose a novel network based on attention mechanism to fuse sensors in a self-motivated and meaningful manner. We design spatial and temporal branches that focus on pairwise images and a sequence of images respectively. Meanwhile, a tiny but effective module (referred to as "warm start") is introduced to produce velocity-related information for the IMU encoder. The proposed attention branches and warm start are shown to improve the robustness of the model in dynamic scenarios and in the case of rapid changes in vehicle velocity. Evaluation on KITTI and Malaga datasets shows that our method outperforms other recent state-of-the-art VO/VIO methods.