Document Image Dewarping and Illumination Correction using Reference Templates

Abstract

In today's fast-paced world, the digitalization of business workflows has become indispensable for organizations to remain competitive and efficient. Documents play a critical role in these workflows, as they contain vital information necessary for decision-making and record-keeping. Despite advancements in digitalization, a significant proportion of documents still exists in physical formats, necessitating digitization for seamless integration into digital workflows. Since manual digitization is labor-intensive and scanner-based digitization inflexible, there is a need for automated document analysis systems that are capable of processing camera-captured document images. Although camera-based digitization offers greater flexibility, it poses significant challenges due to distortions caused by camera angles, document conditions, and varying lighting environments. In this work, we address the problems of document image dewarping and illumination correction, as they are essential preprocessing steps for document analysis. We aim to enhance document images to achieve a scan-like quality, thereby enhancing downstream tasks such as text detection and document understanding. Although the state-of-the-art methods have made significant progress in this area, further advancements are still needed, especially in real-world scenarios. We work towards improving the existing methods by leveraging additional information about the document structure and visual appearance, which we refer to as reference templates. The main contributions of this work are as follows: 1. We create the first large-scale, high-resolution dataset for document image dewarping and illumination correction with reference templates, enabling the development of more accurate and robust document image enhancement models. 2. We introduce two novel deep-learning-based systems for geometric dewarping, which integrate reference templates to minimize distortions in warped document images and thereby significantly improve the quality of the dewarped images. 3. We present a new method for illumination correction of document images using reference templates, thus improving the readability and interpretability of the documents. The contributions are evaluated individually, following predefined requirements and adhering to state-of-the-art evaluation methodologies. The outcome led us to conclude that the information contained in reference templates can be effectively leveraged to improve geometric dewarping and illumination correction. Thereby, we narrow the gap between research and real-world applications, bringing us closer to achieving fully automated document analysis in real-world contexts.