Device-free gesture tracking using acoustic signals

Abstract

In this demo, we present LLAP, a hand tracking system that uses ultrasound to localize the hand of the user to enable device-free gesture inputs. LLAP utilizes speakers and microphones on Commercial-Off-The-Shelf (COTS) mobile devices to play and record sound waves that are inaudible to humans. By measuring the phase of the sound signal reflected by the hands or fingers of the user, we can accurately measure the gesture movements. With a single pair of speaker/microphone, LLAP can track hand movement with accuracy of 3.5 mm. For devices with two microphones, LLAP enables drawing-in-the air capability with tracking accuracy of 4.6 mm. Moreover, the latency for LLAP is smaller than 15 ms for both the Android and the iOS platforms so that LLAP can be used for real-time applications.