Research On Automatic Grading Evaluation Algorithm Of Facial Paralysis Based On Face Images

Facial paralysis is a common functional disorder of the facial muscles,characterized by paralysis of the facial expression muscles on the affected side,distortion of the corners of the mouth,and loss of the nasolabial fold,which often affects the patient’s speech and facial appearance to varying degrees.The early evaluation of the degree of facial paralysis in patients is of vital importance for the accurate formulation of treatment plans and subsequent treatment of facial paralysis.Currently,the assessment of facial paralysis relies mainly on the subjective judgment of professional physicians and facial paralysis scales.These methods are highly dependent on the medical expertise of the physician and may lead to misdiagnosis or missed diagnosis,especially for inexperienced doctors.The evaluation results often have a significant degree of subjectivity and irreproducibility.Automatic assessment of facial paralysis based on computer vision technology using static images and dynamic videos can overcome the limitations of traditional manual methods to a certain extent.However,there are still several challenges in the field of automatic facial paralysis assessment.1)Limited by the scarcity of facial paralysis dataset,difficulty in data acquisition and annotation,the performance of automatic evaluation model has not yet reached expectations.2)Facial asymmetry and facial muscle movement ability are important criteria for facial paralysis grading evaluation,but most existing dynamic video-based evaluation methods focus on exploring inter-frame relationships to model muscle movement ability in the temporal domain,while facial asymmetry features are neglected.In this thesis,to address the challenges of existing methods,the Facial Nerve Palsy Dataset(FNPD)is constructed,and then around this dataset,the following research work is carried out from both static images and dynamic videos,respectively:1.To address the problem of scarce data,difficulty in data acquisition and annotation in the facial paralysis image dataset,and considering the strong similarity in facial expression between the facial expression dataset and the facial paralysis dataset,this thesis proposes a facial paralysis automatic evaluation algorithm based on transfer learning.Firstly,a highly efficient facial feature extractor is pre-trained using a large-scale facial expression dataset to obtain effective representation of facial features.Subsequently,a joint loss function is constructed based on continual learning theory to alleviate catastrophic forgetting during the transfer from the expression source domain to the facial paralysis target domain.Finally,through ablation experiments and comparative experiments with existing facial paralysis automatic grading evaluation methods,the effectiveness of this method is verified.In particular,this method achieved accuracy of 88.5% and F1 score of 85.0% in the task of automatic evaluation of facial paralysis static images.2.In response to the issue of current dynamic video-based facial paralysis evaluation methods that only consider inter-frame correlations while ignoring facial asymmetry in patients with facial paralysis,this thesis proposes a facial paralysis automatic evaluation method based on face region division and a twin-tower structure with a difference network.By constructing a lightweight face key point localization algorithm,Region of Interest(ROI)areas are detected and extracted from input video frame sequences.Subsequently,the corresponding areas and their mirrors are fed into the twin-network branches to extract deep action feature information under the constraint of 3D convolutional autoencoder.Meanwhile,the structural similarity is used to calculate the difference matrix of the decoder,which is used to assist the latent representation extracted by the encoder for facial paralysis automatic evaluation.Finally,through ablation experiments and comparative experiments with existing facial paralysis action recognition models,the effectiveness is verified.Specifically,this approach achieved accuracy of 95.1% and F1 score of 97.6% in facial paralysis video automatic evaluation.3.This thesis integrates two types of facial paralysis evaluation methods under static and dynamic inputs into a mobile phone application.The application is characterized by its simplicity,ease of use,and fast detection,which can assist doctors and facial paralysis patients in completing severity assessment tasks.The design concept is to use static images and dynamic videos as inputs,and infer the degree of illness through two separate branch networks,convert the results into scores using facial paralysis evaluation scales,and finally weight the static images and dynamic videos to intuitively display the severity of the patient’s illness.The significance of this application lies in improving the efficiency of doctors’ evaluations,helping patients monitor their conditions,and assisting doctors in formulating relevant rehabilitation plans to better help patients complete their rehabilitation treatments.