A Real-Time FPGA Implementation of Visible/Near Infrared Fusion Based Image Enhancement

Abstract

Near-infrared (NIR) band sensors capture achromatic images that contain complementary details of a scene which are diminished in visible (VS) band images when the scene is obscured by haze, mist, or fog. To exploit these complementary details, special cameras are produced that can capture both VS and NIR bands. Additionally, several sophisticated VS-NIR fusion approaches have been proposed to produce enhanced VS image with high quality. However, these approaches usually encounter heavy computations that are performed off cameras, limiting its suitableness for real-time applications. In this paper, we present an efficient design and FPGA implementation of VS-NIR fusion based image enhancement approach. To be suitable for camera hardware integration and to meet video real-time requirements, the proposed implementation is designed such that the consumed FPGA resources and the number of clock cycles required for producing the fused image are kept small. To achieve this, data and calculations are reused whenever possible in addition to the parallel and pipelined operations at both data and task levels. The proposed implementation is synthesized and validated on a low-end FPGA device. The proposed implementation produces a real time, indistinguishable, and 5 times faster fused image compared with optimized C++ implementation executed on a modern computer.