Depth-guided Deformable Convolutions for RGB-D Saliency Object Detection

Abstract

Recently, RGB-D salient object detection(SOD) has attracted increasing research interests, and existing methods have achieved huge success owing to well-designed feature extraction and fusion. However, in existing methods, the depth maps cannot be utilized entirely since RGB and depth are usually concatenated together as an entirety and then feed into the backbone to extract features, which cannot achieve the spatial supervision between both modals. In this letter, we propose a Depth-guided Deformable 3D Convolution (Guided-Conv) to solve this problem. Specifically, the Guided-Conv obtains the sampling offset of the 3D convolution kernel guided by the extra depth input, enabling the convolutional layer to change the receptive field and adapt to geometric cross-modal transformations. Besides, the Guided-Conv also incorporates geometric cues into the forward propagation by producing spatially adaptive filter weights. Based on comprehensive experiments on several extensively used bench-marks, the Guided-Conv yields strong results against several state-of-the-art RGB-D SOD approaches based on four key evaluation metrics.