Automated Testing Method for Font on GUI Based on Scene Text Detection and Font Recognition

Abstract

The font is a basic property of software GUI elements. A variety of fonts improve users' visual experience and specific fonts play a significant highlight role for the text content of some software. However, font configuration is very error-prone during software development and the error is difficult to discover sometimes. Font misconfiguration in GUI impacts the usability of the software and even causes infringement or violation of laws and regulations. To the best of our knowledge, there is no related work on how to efficiently check the problem so far. We propose an automated testing method for GUI fonts for the first time. We design and implement an end-to-end complete process for GUI font automated testing. Image patches of GUI text element area are detected and extracted by the EAST model and then input into the trained font recognition model to predict the font type. The prediction font type is then compared with the expected one to figure out whether the font is right. Besides, we create a dataset that fits the visual feature of GUI font, which facilitates building the font recognition model GoogLenet-1 to obtain an accuracy of over 63% in Top-1 and 90% in Top-5 on the real GUI test set.