Unmanned Aerial Vehicle Control Using Hand Gestures and Neural Networks

Abstract

Neural Networks are a series of data manipulations inspired by how neurons perceive information in the brain. This technology is useful for accomplishing tasks that conventional computers do poorly, but people do accurately. Neural Networks are utilized in this project to control an Unmanned Aerial Vehicle (UAV) with hand gestures. For this case the model produced by TensorFlow will take twenty-one different hand points on a user’s hand using MediaPipe and distinguish which of eight gestures the user is signaling. This data is received through a camera on the UAV and once ran through the model the flight path will be controlled. The hand points are logged as two-dimensional coordinates in relation to the pixel they are in the frame. This creates a model with forty-two inputs and nine outputs. The model can run at around twenty frames per second due to the low number of inputs. The UAV can handle efficiently due to an acceptable processing time of its commands.