Annotating RGBD images of indoor scenes

Abstract

Annotating RGBD images with high quality semantic annotations plays a crucial key to the advanced scene understanding and image manipulation. While the popularity of affordable RGBD sensors has eased the process to acquire RGBD images, annotating them, automatically or manually, is still a challenging task. State-of-the-art annotation tools focus only on 2D operations and provide at most image segmentation and object labels even with the presence of depth data. In this work, we present an interactive system to exploit both color and depth cues and facilitate annotating RGBD images with image and scene level segmentation, object labels and 3D geometry and structures. With our system, the users only have to provide few scribbles to identify object instances and specify the label and support relationships of objects, while the system performs those tedious tasks of segmenting image and estimating the 3D cuboids. We test the system on a subset of benchmark RGBD dataset and demonstrate that our system provides a convenient way to generate a baseline dataset with rich semantic annotations.