Research Of Human-Computer Interaction Based On Gesture Recognition On Android Platform

With the highly development of the mobile Internet,mobile terminals(MT)such as smart phones and Tablet PC of Android platform have gradually become more and more popular in the daily life of public with its high wireless networks transmission speeds,its varied function and convenient use.The ever-expanding device functions led to an increase of users` demand of Human-computer interaction and operational complexity.Gesture which conforms to people's daily operating habits is a method of non-contact information exchange and interaction.Compared with the contact human-computer interaction such as mouse,keyboard and touch screen,it hard to cause the equipment pollution,and it can perform in the long distance.In this dissertation,this dissertation is dedicated to the hand gesture interaction technology based on computer vision on Android platform.Furthermore,an eye-tracking interaction system which has the function of hand gesture recognition by camera and can control mouse are proposed to solve the problems above.The system can satisfy the real-time requirement.This dissertation is dedicated to the transplantation and deployment technology of deep learning on the Android platform.Then the transplantation and deployment technology of Caffe deep learning framework is implemented to run a hand segmentation method based on convolutional neural networks on the Android platform.Meanwhile,in the face of the problem that the computing performance rate of CPU in mobile terminal is not high,we define adaptive gesture detection area and simplify the Convolutional Neural Network framework to accelerate the system operation performance.The system frame rate is increased to 9fps~12fps,which satisfies the real-time requirements of people operations.In terms of face detection,Human face classifier based on the Haar-like feature and Adaboost algorithm is proposed to detect face and a distance-based human eye localization method is proposed.Human eye classifier based on the Haar-like feature is used in short distance,while an eye positioning algorithm based on the geometric features and positional features of eye is used in long distance.This method is simple in calculation and fast in operation.It can satisfy the real-time requirement at high detection accuracy.In terms of hand gesture recognition,a gesture segmentation algorithm based on convolutional neural network is proposed to solve the problems that incomplete gesture segmentation and low accuracy of hand gesture recognition in complex backgrounds environment such as when face and gesture coincide.Experimental results show that this algorithm has a high accuracy of 92.6%,96.0% and 91.4% in five,fist and point gesture when face and gesture coincide partly.On the basement of eye positioning and finger positioning,we design a Human-computer interaction system on the Android platform which based on eye-tracking.This system calculates the mouse position on the screen and control the mouse to move by the human eye coordinates and the fingertip coordinates in the video sequence,spatial relationship between the smart device and user and screen positioning model based on eye-tracking.Furthermore,this dissertation is dedicated to the orthogonal polynominal regression combined with least-square fit and Linear Interpolation to fit the mouse trajectory.The algorithms mentioned above can make the mouse operation more smoothly and improve the friendliness of human-computer interaction.Experiments show that the system can complete the mouse positioning and moving operation with the average distance error of 0.386 cm and 4.193 cm respectively on the human-computer interaction platform of smart phone and smart TV,which improves the accuracy and friendliness of human-computer interaction based on gesture recognition.