Rectification and 3D reconstruction of curved document images

Abstract

Distortions in images of documents, such as the pages of books, adversely affect the performance of optical charac-ter recognition (OCR) systems. Removing such distortions requires the 3D deformation of the document that is often measured using special and precisely calibrated hardware (stereo, laser range scanning or structured light). In this paper, we introduce a new approach that automatically re-constructs the 3D shape and rectifies a deformed text doc-ument from a single image. We first estimate the 2D distor-tion grid in an image by exploiting the line structure and stroke statistics in text documents. This approach does not rely on more noise-sensitive operations such as image bina-rization and character segmentation. The regularity in the text pattern is used to constrain the 2D distortion grid to be a perspective projection of a 3D parallelogram mesh. Based on this constraint, we present a new shape-from-texture method that computes the 3D deformation up to a scale factor using SVD. Unlike previous work, this formu-lation imposes no restrictions on the shape (e.g., a devel-opable surface). The estimated shape is then used to re-move both geometric distortions and photometric (shading) effects in the image. We demonstrate our techniques on doc-uments containing a variety of languages, fonts and sizes. 1.