Research On Flexible Mapping Interaction Model And Algorithm Among Multiple Gestures And One Semantic

In recent years,gesture-based interaction has become a research hotspot in the field of human-computer interaction with its natural and intuitive characteristics.Gesture is one of the important ways of non-verbal communication between people,and it plays an important role in human life.As an input way of human-computer interaction system,gestures can express intentions via various actions.Compared with the traditional interaction ways using keyboard and mouse,interaction with gesture is free from the constraints of traditional interactive devices,and it makes human-computer interaction more natural and harmonious.Therefore,the gesture-based interaction has important research value and application value.In order to solve the problem that the interaction interface change is inconsistent with the expected change due to the error of gesture recognition or unrecognition of gesture during the gesture-based interaction,the flexible mapping interaction model among multiple gestures and one semantic is proposed.And it makes gestured-based interaction smoother,more natural and lower interaction load.In order to solve the problem that gesture recognition rate cannot meet the requirement of interaction.A method for recognition of static gesture and trajectory gesture are proposed,which improve the recognition rate of static gesture and trajectory gesture.And the content and innovations of this paper are as follows:(1)A static gesture recognition method based on area feature combined with the main direction of gesture is proposed.In this paper,the color image of gesture is obtained by Kinect,and it is processed as a background pure static gesture image.Then the area feature of the static gesture is obtained by combining with the main direction of the gesture.By obtaining the area feature of static gesture,the problem of different size and different rotation angle of gesture is solved,and it effectively reduces the influence of the different details of gesture such as different angle between fingers.(2)A method of trajectory gesture recognition based on Convolution Neural Network is proposed.In this paper,the trajectory point coordinates of the trajectory gesture are obtained by Kinect.The curve of the first degree is selected to fit the trajectory point into a trajectory curve,and it means the trajectory gesture is converted into image of trajectory gesture for recognition.The Convolutional Neural Network method is used to recognition the trajectory gestures and the AlexNet is selected as the network structure.Then a set containing 4500 samples for each trajectory gesture is established.Finally,the trained network mode is used to recognition the trajectory gestures.The method improves the trajectory gesture recognition rate,and the method has a good real-time performance.(3)The flexible mapping interaction model among multiple gestures and one semantic is proposed.In order to solve the problem that the interaction interface change is inconsistent with the expected change due to the error of gesture recognition or unrecognition of gesture during the gesture-based interaction,the flexible mapping interaction model among multiple gestures and one semantic is proposed.And it is demonstrated that it conforms to the general behavior habits of human beings from the perspective of cognitive basis.The common features of multiple interactive gestures which corresponding to one semantic is quantitatively analyzed.And the flexible mapping interaction model is evaluated from four aspects of learning,naturalness,mental load and operability.Finally,the solution to the problem that the interaction interface change is inconsistent with the expected change due to the error of gesture recognition or unrecognition of gesture during the gesture-based interaction is summarized.