Dense pose estimation of deformable objects

Abstract

Statistical Deformable Models (SDMs) represent a widely used collection of computer vision techniques. SDMs serve the purpose of modeling the deformation and variability of a specific class of objects, e.g. faces, ears, hands, human poses, etc. SDMs have shown success in many areas of visual image analysis. For example, in biometrics, SDMs are commonly used for recognition, classification, detection, and alignment. During the past few years, we have witnessed the development of many methodologies for building and fitting Statistical Deformable Models. The construction of accurate SDMs requires careful annotation of images with regards to a consistent set of landmarks. However, the manual annotation of a large number of images is a tedious, laborious and expensive procedure. Furthermore, for several deformable objects, e.g. the human body, it is difficult to define a consistent set of landmarks. Even though, the SDMs construction still suffers in terms of sparse landmark annotation. With sparse annotations, the shape deformation of any deformable objects is only captured where the key points are labeled, thus nuanced shape deformation is discarded. It is merely possible to accurately annotate dense shape deforestation manually. Fortunately, for the majority of objects, it is possible to extract the shape by object segmentation or even by shape drawing. In this thesis, we propose that it is possible to construct SDMs by putting objects shapes in dense correspondence. Such SDMs can be built with much less effort for a large battery of objects. Additionally, by sampling the dense model, a part-based SDM can be learned with its parts being in correspondence. We propose a framework to develop SDMs of human arms and legs, which can be used for the segmentation of the outline of the human body, as well as to provide better and more consistent annotations for articulated objects. Such a system can serve as an element of cascaded architectures that jointly localize landmarks and estimate dense correspondences. Also, we show that the obtained dense correspondence can act as a source of prior knowledge that complements and extends the pure landmark-level annotations, accelerating and improving the training of pose estimation networks. The thesis contributes towards fully automatic dense correspondence estimation. Such correspondence would benefit any system that requires nuance shape deformation. We are approaching a milestone that dense correspondence and poses for deformable objects can be estimated “in-the-wild” with minimal effort on labeling. The estimated correspondence is shown to be useful for various downstream tasks for a collection of objects.