Registration-Based Stereo Rectification

Abstract

In stereo depth prediction, stereo calibration and rectification is the act of determining the rotation and translation between a stereo pair, and then using the two values to appropriately align the images such that all point correspondences lie on the same horizontal line. Without this crucial step, most stereo depth prediction algorithms fail. Most classical methods have relied solely on feature points and matches to determine these extrinsic values. These feature point matches can come from handmade physical markers, or from feature detectors such as SIFT or SURF. In this paper, we propose a novel unsupervised featureless warping based method for stereo rectification. Our method directly optimizes over the rotation and translation values to arrive at one where the images are best aligned. Furthermore, we also use neural networks to parametrize the rotation and translation, thereby learning the extrinsic parameters of a stereo pair. Lastly, we test our method on real world datasets like KITTI to demonstrate its accuracy and utility.