TinyML for Real-Time Embedded HD-EMG Hand Gesture Recognition with On-Device Fine-Tuning

Abstract

This work introduces a fully embedded wireless platform that incorporates the Coral Tensor Processing Unit (TPU) accelerator to leverage TinyML for real-time hand gesture recognition using high-density surface electromyography (HD-sEMG). With a general inference time of 2.96 ms using a 64 channels sensor, the TPU proved to be well suited for such real-time recognition tasks. Constructed from off-the-shelf components, the platform offers a cost-effective and self-sufficient alternative for integrating artificial intelligence into prosthetic devices, eliminating the dependency on expensive external hardware. The system allows for intuitive calibration through a user interface, facilitating fine-tuning of the inference model directly on the device or remotely via a cloud-based server. On-device finetuning yields similar performance to the cloud-based approach, improving gesture recognition accuracy by up to 36.15% in intersession test cases. Extensive exploration of 8-bit data quantization techniques demonstrates that hardware compatibility can be achieved without sacrificing performance. In the best case, the proposed quantization scheme can improve the results by 0.96% compared to unquantized data. Overall, this paper establishes a robust foundation for advancing on-device HD-sEMG based hand gesture recognition, paving the way for more accessible and practical myoelectric prosthetic solutions.