Modality-Guided Subnetwork for Salient Object Detection

Abstract

Recent RGBD-based models for saliency detection have attracted research\nattention. The depth clues such as boundary clues, surface normal, shape\nattribute, etc., contribute to the identification of salient objects with\ncomplicated scenarios. However, most RGBD networks require multi-modalities\nfrom the input side and feed them separately through a two-stream design, which\ninevitably results in extra costs on depth sensors and computation. To tackle\nthese inconveniences, we present in this paper a novel fusion design named\nmodality-guided subnetwork (MGSnet). It has the following superior designs: 1)\nOur model works for both RGB and RGBD data, and dynamically estimating depth if\nnot available. Taking the inner workings of depth-prediction networks into\naccount, we propose to estimate the pseudo-geometry maps from RGB input -\nessentially mimicking the multi-modality input. 2) Our MGSnet for RGB SOD\nresults in real-time inference but achieves state-of-the-art performance\ncompared to other RGB models. 3) The flexible and lightweight design of MGS\nfacilitates the integration into RGBD two-streaming models. The introduced\nfusion design enables a cross-modality interaction to enable further progress\nbut with a minimal cost.\n