TLWSR: Weakly supervised real‐world scene text image super‐resolution using text label

Abstract

Abstract Scene text image super‐resolution (STISR) has recently received considerable attention. Existing STISR methods are applicable to the situation that all the LR‐HR pairs are available. However, in real‐world scenarios, it is difficult and expensive to collect ground‐truth HR labels and align them with LR images, and thus it is essential to find a way to implement weakly supervised learning. We investigate the STISR problem in the situation that only a subset of HR labels is available and design a weak supervision framework using coarse‐grained text labels named TLWSR, which combines incomplete supervision and inexact supervision. Specifically, a lightweight text recognition network and connectionist temporal classification loss are used to guide the super‐resolution of text images during training. Extensive experiments on the benchmark TextZoom demonstrate that TLWSR generates distinguishable text images and exceeds the fully supervised baseline TSRN in boosting text recognition accuracywith only 50% HR labels available. Meanwhile, TLWSR can be applied to different super‐resolution backbones and significantly improves their performance. Furthermore, TLWSR shows good generalization capability to low‐quality images on scene text recognition benchmarks, which verifies the effectiveness of this framework. To the authors' knowledge, this is the first work exploring the problem of STISR in weakly supervised scenarios.