Adaptation via Proxy: Building Instance-Aware Proxy for Unsupervised Domain Adaptive 3D Object Detection

Abstract

3D detection task plays a crucial role in the perception system of intelligent vehicles. LiDAR-based 3D detectors perform well on particular autonomous driving benchmarks, but may poorly generalize to other domains. Existing 3D domain adaptive detection methods usually require annotation-related statistics or continuous refinement of pseudo-labels. The former is not always feasible for practical applications, while the latter lacks sufficient accurate supervision. In this work, we propose a novel unsupervised domain adaptive framework, namely A daptation V ia P roxy ( AVP ), that explicitly leverages cross-domain relationships to generate adequate high-quality samples, thus mitigating domain shifts for existing LiDAR-based 3D detectors. Specifically, we first train the detector on source domain with the curriculum example mining (CEM) strategy to enhance its generalization capability. Then, we integrate the profitable instance knowledge from the source domain with the contextual information from the target domain, to construct the instance-aware proxy, which is a data collection with diverse training scenes and stronger supervision. Finally, we fine-tune the pre-trained detector on the proxy data for further optimizing the detector to overcome domain gaps. To build the instance-aware proxy, two components are proposed, i.e. , the multi-view multi-scale aggregation (MMA) method for producing high-quality pseudo-labels, and the hybrid instance augmentation (HIA) technique for integrating the knowledge from source annotations to enhance supervision. Note that AVP is architecture-agnostic thus it can be easily injected with any LiDAR-based 3D detectors. Extensive experiments on Waymo, nuScenes, KITTI and Lyft demonstrate the superiority of the proposed method over the state-of-the-art approaches for different adaptation scenarios.