Multispectral pedestrian detection based on deep convolutional neural networks

Abstract

Vision-based pedestrian detection for all day are crucial in Advance Driver Assistance Systems (ADAS), autonomous vehicles and video surveillance. Based on the fact that human body radiation falls around 9.3pm, thermal images have distinctive advantages in pedestrian detection at nighttime. With the recent success of CNNs in vision community, how to properly explore information in color and thermal images in CNNs-based methods attracts attention of researchers. The contributions of this paper are twofold: First, multiple multispectral pedestrian detectors based on the Single Shot Detector (SSD) framework have been developed. Second, the performance of different pixel-level image fusion methods in multispectral CNN-based pedestrian detectors is evaluated. Extensive results based on KAIST multispectral pedestrian benchmark show that good pixel-level image fusion methods are complementary to both early-fusion and late-fusion CNN architectures at nighttime. The combination of image fusion and late-fusion CNN architectures can more properly exploit the multispectral information and achieve the best detection performance.