Text detection in natural scenes: a literature review

Abstract

Text detection is widely applied in the automatic driving and cross-modal image retrieval fields. This technique is also an important pre-procedure in optical character-based text recognition tasks. At present, text detection in complex natural scenes remains a challenging topic. Because text distribution and orientation are varied in different scenes and domains, there is still room for improvement in existing computer vision-based text detection methods. To complicate matters, natural scene texts, such as those in guideposts and shop signs, always contain words in different languages. Even characters are missing from some natural scene texts. These circumstances present more difficulties for feature extraction and feature description, thereby weakening the detectability of existing computer vision and image processing methods. In this context, text detection applications in natural scenes were summarized in this paper, the classical and newly presented techniques were reviewed, and the research progress and status were analyzed. First, the definitions of natural scene text detection and associated concepts were provided based on an analysis of the main characteristics of this problem. In addition, the classic natural scene text detection technologies, such as connected component analysis-based methods and sliding detection window-based methods, were introduced comprehensively. These methods were also compared and discussed. Furthermore, common deep learning models for scene text detection of the past decade were also reviewed. We divided these models into two main categories: region proposal-based models and segmentation-based models. Accordingly, the typical detection and semantic segmentation frameworks, including Faster R-CNN, SSD, Mask R-CNN, FCN, and FCIS, were integrated in the deep learning methods reviewed in this section. Moreover, hybrid algorithms that use region proposal ideas and segmentation strategies were also analyzed. As a supplement, several end-to-end text recognition strategies that can automatically identify characters in natural scenes were elucidated. Finally, possible research directions and prospects in this field were analyzed and discussed.