Device-Free Handwritten Character Recognition Method Using Acoustic Signal

Abstract

In this paper, we propose a method for recognizing handwritten characters by a finger using acoustic signals. This method is carried out using a smartphone placed on a flat surface, such as a desk. Specifically, this method uses an ultrasonic wave transmitted from the smartphone, which is reflected by the finger, and an audible sound is generated when writing a handwritten character. The proposed method does not require an additional device for handwritten character recognition because it uses the microphone/speaker built into the device. Evaluation results showed that it was able to recognize 36 types of characters with an average accuracy of 77.8% in a low noise environment for 10 subjects. In addition, it was verified that combining an audible sound and an ultrasonic wave in this method achieved higher recognition accuracy than when only an audible sound or an ultrasonic wave was used.