Real-Time FPGA-Based Binocular Stereo Vision System with Semi-Global Matching Algorithm

Abstract

Binocular stereo vision, which is an attractive topic in the computer vision area, mainly focuses on extracting depth information through a stereo matching algorithm. The semi-global matching algorithm (SGM) is well-studied but is hard to be applied in real-time because of the high computational cost. In this work, we report an FPGA-based stereo vision system for the SGM. The system contains a camera controller, a module for image rectification, and the SGM accelerator. The SGM accelerator module can extract the depth information with very high speed by a parallel pixel-level pipeline architecture. The left-right check module further ensures the quality of the disparity map. We implement the system on the Stratix-V platform with a 14.9% error rate and it consumes about 58K LUTs, 59K registers, and 5.2M bits of on-chip memory. The maximum working frequency can reach up to 135.96 MHz for the XGA (1024×768) video and 128 disparity range with the power dissipation of 4.302 W and the processing speed of 173 XGA-frames per second.