Drivable Area Segmentation Based on HybridNets and Text Prompt Network

Abstract

Drivable area segmentation and traffic object detection are important prerequisites for autonomous navigation of intelligent vehicles. Recently, multi-task network is used for segmentation and detection task because of its high efficiency brought by multiplexing features. However, the existing benchmarks mostly perform poorly when the top-level tasks show significant differences. Inspired by text prompt network from Natural Language Processing, we propose a text-prompt-based feature enhancement optimization method of object detection and drivable area segmentation in traffic scene. We design text prompt and fuse it with image feature. The end-to-end multi-task neural network is used to realize the object detection with higher precision and the drivable area segmentation based on traffic rule semantics with small data volume. Experimental results show that our proposed text-prompt feature enhancement approach on HybridNets achieves competitive performance on Gazebo simulation scene.