Non-Rigid Structure from Motion through Estimation of Blend Shapes

Abstract

In this paper, we propose a prior-free approach to estimate non-rigid object from 2D image trajectories assuming the affine camera model. As mentioned in some recent works [7, 8], most low- rank methods are unable to recover objects with complex motion. We identify the small deformation condition as the condition fundamental to the triple column metric upgrade algorithm commonly used in many low-rank methods, and accordingly modify this algorithm so that it becomes independent of the number of basis. Inspired by the blend shape technique used in computer graphics, we model the non-rigid object as a combination of blend shapes. Unlike many existing methods that estimate an average shape plus a few directions of deformation, we recover each blend shape as a valid 3D shape through the introduction of a pseudo view, which helps to prevent degeneration in the direction of the camera axes. This gives the blend shapes clear physical meaning, and makes the method robust against overfitting. Experiments on synthetic datasets and real tracking datasets show that the proposed method outperforms the existing methods in both 3D error and robustness.