Interactive Rehabilitation Training System Based On Facial Expression Recognition

Facial Expression Recognition(FER)is the process of emotional recognition of face based on affective cognition theory.Facial expressions can convey extensive information in daily communication.For example,If you understand the emotional state of the communicator,the dialogue can be smoother.Likewise,the Human-Computer Interaction(HCI)system,which is based on the development of computers and algorithms,can enhance the degree of immersion and improve the HCI experience significantly.Therefore,the rehabilitation of FER-based HCI can improve the affective cognitive ability of people with affective cognitive impairment.Given that,this paper studies the FER algorithm with real-time and computational complexity,and combines FER with the cognitive characteristics of patients who affective cognitive impairment to build an HCI rehabilitation training system.The contents are divided into the following three aspects:Firstly,proposing a convolutional neural network(CNN)architecture based on depth-wise separable convolution layers,multi-scale information inception and feature mosaic operation,which named “Concat_Xception” lightweight network.The network architecture using depth-wise separable convolution layers and global average pool layers instead of full connection layer to reduce parameters.The feature channel weight values are learned after feature fusing to obtain more characteristic features,and the global information aggregation is introduced for softmax classifiers.Then Concat_Xception network achieved an accuracy of 70.13% in the facial expression recognition challenge dataset(Fer2013).Secondly,based on Concat_Xception network,a real-time FER system is built on a low-power platform in a natural scene.In the FER system,face recognition is implemented by the Histogram of Oriented Gradient(HOG)algorithm,then using Concat_Xception network to realize the expression recognition.Finally,the FER system obtains 6.52 fps on the NVIDIA Jetson TX2 platform,which meets the requirements of real-time.Thirdly,combining the FER system with the affective cognition characteristics of people who has affective cognitive impairment,such as Autism Spectrum Disorder(ASD),proposed the interactive rehabilitation paradigm.Reference to the therapists,we designed a emotional inducement experiments to analyze the expression characteristics of ASD.Then combined FER system with ASD emotional intervention course proposed a rehabilitation training paradigm,deployed in Jeston TX2 and Android,in order to assist therapists in rehabilitation training.The research on the emotional rehabilitation training,the feasibility of the rehabilitation paradigm is verified by analyzing the interaction test data of normal people.Furthermore,the visual interface for facial expression and the feedback signal of the system can improve the enthusiasm of the interactor.