Machine Learning Based Gesture And Expression Recognition Technology And Its Application In Human-machine Interaction

Physical interaction and emotional interaction are indispensable in natural humanmachine interaction.Gesture is one of the most flexible,free,expressive and least limited body movements,which is widely used in the physical interaction of human-machine interaction.Surface Electromyography(s EMG)is an important biological signal produced by muscle contraction,which directly reflects the intention of human body movement and is not affected by environmental factors.Therefore,gesture recognition based on s EMG has become a hot topic in the research of natural human-machine interaction.Automatic Facial expression recognition(FER)enables robots to understand human emotional changes,which is conducive to the emotional health of users and has become an important part of human-machine interaction.However,although most commercial or laborator-developed emg acquisition systems can transmit data wirelessly,such devices have fewer channels available or are less flexible,portable or gesture sensitive.In addition,the FER task in complex natural scenarios remains a challenging problem.Aiming at the above problems,this paper will study the gesture and expression recognition technology and its application in natural human-machine interaction.The main work includes:(1)Design the overall scheme of gesture sensing,gesture recognition and dexterous hand control system.Gesture sensing based on s EMG was constructed.Design and implementation of smart wearable armband(SWA).Test SWA for upper limb of s EMG signal acquisition,preprocessing,and wireless transmission function,validation of this study is to build based on the multi-channel s EMG signal of the sensor,the feasibility and effectiveness of and SWA is small and light,soft,comfortable,low power consumption characteristics for the following s EMG signal feature extraction,feature selection,one hand gesture recognition based on machine learning and natural gesture interaction application foundation.(2)Based on the gesture sensing constructed in this study,s EMG signal feature optimization,classifier design,real-time gesture recognition and dexterous hand control are studied.The effects of 9 machine learning algorithms and 5 normalization methods on performance evaluation indexes of 6 classifiers are studied.Different electrode acquisition locations and number of sensor channels were studied.Genetic algorithm was used to optimize the characteristics.9 kinds of gestures mean offline recognition accuracy and real-time recognition accuracy were 99.88% and 96.20% respectively,five fingers dexterous hand real-time accurate imitation user gestures,verify the reliability of gesture sensing system,the gesture recognition algorithm based on machine learning efficiency and gesture recognition technology of gesture interaction in the natural human machine interaction application practicability.(3)In the aspect of expression recognition and expression driving for natural humanmachine interaction,a deep network model based on c GAN was proposed for facial expression recognition aiming at the sample imbalance of expression data set in complex natural scenes.Cyclic consistency loss is used to achieve identity ID invariance and solve the problem of unreachable paired data in expression data set under complex natural scenes.An optimization strategy for optimizing four loss functions is proposed to decouple the face emotional representation learned by the model from other variables(such as identity,head position and illumination).The experimental results of the proposed expression recognition algorithm based on c GAN on two public expression recognition data sets(Affect Net and RAF-DB)were evaluated and analyzed to verify the effectiveness of the expression recognition algorithm based on machine learning.To capture and analyze the intensity of facial expression motion as the facial motion unit(AUs),design the mapping relationship between the motion unit(AUs)and the Unity3 D deformation target(MTs),and verify the feasibility of using the motion unit(AUs)mapping as the deformation target(MTs)and driving the expression of the Unity3 D virtual character in the natural human-machine interaction.