Implementation of Hand Gesture Recognition System Based on Leap Motion and Neural Network

Abstract

With the development of technology the interaction between human and robot is gradually changing with primary design to be simple and intuitive In recent years the research of mobile robot has been widely discussed and especially the focusing on the mobile robots as the remote controller cars and as Unmanned Aerial Vehicle (UAV) are the most popular topics One of the purposes of the mobile robots is their application in hazardous environments including rescue survey terrain and tracking In fact the traditional technology usually applied remote controller and computer commands to control the mobile robots In addition for inexperienced users it is difficult to handle the robot's direction speed and attitude Consequently this research has proposed a gesture controlled device with simple and easy operation to reduce the training time in exerting the remote controller This thesis is divided into three parts; namely somatosensory sensor software interface and recognition algorithm The somatosensory sensor adopts Leap Motion that applies the infrared sensor and the CMOS camera to capture basic hand information like finger coordinate velocity direction angle and etc With respect to the software interface the Graphical User Interface (GUI) of the software “Visual Studio 2012” is developed with the C sharp (C#) platform By combining the GUI with SDK provided by Leap Motion gesture recognition is thus designed and displayed on the user interface platform By using Back-Propagation Neural Network (BPNN) to learn and identify the recognition algorithm is developed to recognize the hand gesture numbers 0 1 2 3…9 Moreover the hand features including finger length and angle were extracted Then with the right-hand and left-hand identification results combinations of gestures made by controlling the mobile machines were obtained with the number up to 99 gestures This thesis has therefore proposed development and design of the feature recognition system and instant gesture control system through integrating the features of Leap Motion hand detection remote control robot and C sharp

Date of Award	2015 Jul 27
Original language	English
Supervisor	Teh-Lu Liao (Supervisor)