Category-Level Metric Scale Object Shape and Pose Estimation

Abstract

Advances in deep learning recognition have led to accurate object detection\nwith 2D images. However, these 2D perception methods are insufficient for\ncomplete 3D world information. Concurrently, advanced 3D shape estimation\napproaches focus on the shape itself, without considering metric scale. These\nmethods cannot determine the accurate location and orientation of objects. To\ntackle this problem, we propose a framework that jointly estimates a metric\nscale shape and pose from a single RGB image. Our framework has two branches:\nthe Metric Scale Object Shape branch (MSOS) and the Normalized Object\nCoordinate Space branch (NOCS). The MSOS branch estimates the metric scale\nshape observed in the camera coordinates. The NOCS branch predicts the\nnormalized object coordinate space (NOCS) map and performs similarity\ntransformation with the rendered depth map from a predicted metric scale mesh\nto obtain 6d pose and size. Additionally, we introduce the Normalized Object\nCenter Estimation (NOCE) to estimate the geometrically aligned distance from\nthe camera to the object center. We validated our method on both synthetic and\nreal-world datasets to evaluate category-level object pose and shape.\n