A Virtual Reality-based Rehabilitation Training And Quantitative Evaluation System For Facial Paralysis

Facial paralysis mainly occurs on one side of the face,and affects facial movement function.It brings serious negative effects to the patient’s physical and mental health,and greatly affects the patient’s daily life and work.Facial muscle exercise can effectively recover facial nerve function,and repeated exercise of facial muscles can promote the recovery of facial muscle motor function and improve the rehabilitation effect of facial paralysis.The traditional facial muscle exercise method is mirror biofeedback facial exercises.This exercise is tedious and the patient is not motivated.Meanwhile,treatment needs to evaluate the patient’s facial nerve function.Most existing evaluation methods are manually evaluated by doctors according to the scale,which are time-consuming and laborious,and subjective.To address these problems,this paper presents a facial paralysis rehabilitation training and quantitative evaluation system based on virtual reality that can help patients with facial paralysis exercise and automated quantitative evaluation.The system consists of an exercise module and an evaluation module.Exercise module: The purpose of the exercise module is to improve the enthusiasm of the patient’s facial muscle exercise and to restore facial nerve function through repeated exercise of the facial muscle.In the exercise,the patient faces the front-facing camera of the mobile phone,and follows the animation prompts to make six facial actions of resting,raising brows lift,frowning,closing eyes,smile,and lip pucker.Except for resting,each action lasts for 3 seconds,with an interval of 3 seconds,and repeats 5 times.First,the system captures real-time images that the patient exercises facial actions according to the animation prompts.Secondly,the system acquires the coordinates of the facial feature points via face alignment.Thirdly,inclination correction and coordinate translation are performed to facilitate subsequent calculation.Then,calculates the completion degree of facial action and determines whether the motion is completed according to the defined level of difficulty.Finally,outputs the completion degree of the action.Evaluation module: The evaluation module can automatically evaluate the facial paralysis patient and output the House-Brackmann facial nerve grading system(HBGS)grade.First,the image preprocessing is performed.Cut out the image with the maximum range of facial movements of raising brows lift,closing eyes and smile is captured from the input video,and then acquire the ROI via face alignment,and crop the image to the same size.All the pictures are divided into three groups of actions,namely,raising brows lift,closing eyes and smile.We designed a dual-channel network.In the network,two networks,VGG19 and Res Net50,are used as the feature extraction part.Input two images of raising brows lift,closing eyes or smile with the same HBGS grade.The two network models are combined to generate a feature,and softmax is obtained after global average pooling.Finally,the HBGS grade is output.The system was validated in 102 facial paralysis patients with different HBGS grades and 9 healthy volunteers.The results show that the HBGS grade obtained from the output of the dual-channel network model we designed has a precision of 95.9%compared with the result of manual evaluation by doctors as the gold standard,which is better than previous reports.Moreover,the system can also improve the patient’s enthusiasm for facial muscle rehabilitation training,and the method is simple and quick,which is helpful for facial paralysis rehabilitation.