An Interactive Sentiment Analysis In Severely Paralyzed Patients Based On Deep Learning

Paralysis is a common disease,the severely paralyzed patient cannot take care of themselves,which brings a heavy burden to patients' psychology,family and society.Many reasons can cause paralysis,such as stroke,traffic accident,child cerebral palsy,spinal injury.There is a special group in the severely paralyzed patient,who cannot move their limb and body and has language barriers,but the patient has clear consciousness and can make head and face movements.The head movement refers to the head moving up,down,left and right,the face movement refers to blinking of the eyes and opening and closing of the mouth.According to the statistical results from a hospital in Shandong,the proportion of the special patient in the severely paralyzed patient is about 10%.If the system can obtain the patient's real intention,the patient can own self-care ability through the rehabilitation mechanical device.The main content of this research is to analyze the real intention of the patient through video,and to develop a corresponding software system.Its purpose is to make the patient has self-care ability,while reducing the burden of patients' family.In the research,there are both active and passive ways to obtain the real intention of the patient.Active way means that the patient expresses their intention consciously.When the patient is in a non-emergency situation,the brain consciousness of the patient is sober,according to the characteristic that the patient's head and face is moveable,the patient can express demands to the system through the head and face movements,and then the system uses the auxiliary device to meet the basic demands of the patient(drinking water,eating,taking medicine,etc.).Passive way means that the system analyses the patient's intention using the patient's expression.When the patient is in an emergency situation,maybe the patient cannot express demand by themselves,the system judges the patient's emotional state according to the patient's facial expression,and then figures out the patient's real needs according to the patient's emotional state,then the system will show the result to the patient.Within a certain amount of time,if the patient can complete the response through head and face movements,the system will perform corresponding operation;if the patient cannot complete the response,the system will inform the patient's family or medical staff.This work includes the following aspects:(1)Head movement recognition based on regionalization.The system detects the patient's face in the patient's head image by the Adaboost algorithm,and recognizes the head movement by the offset direction of the face's center position.(2)Face movement recognition based on Uniform Local Binary Pattern(Uniform LBP)texture feature map.Firstly,the system cuts out the patient's eye and mouth regions in the face image according to the geometrical feature of the face;secondly,the system calculates the Uniform LBP texture feature map of the eye and mouth region images,and the statistical histogram of the feature map is calculated;finally,the system identifies the face movement using the k-Nearest Neighbor method.(3)Facial expression recognition based on deep convolutional neural networks.Firstly,the system increases the number of the samples by the performance(slant correction and image rotation);secondly,the system normalizes the sample image;finally,the system trains three expression recognition models of the patient through the convolutional neural networks in deep learning.When the system recognizes the patient's expression,the system classifies the expression coarsely,and then classifies the expression finely.The main contributions of this paper are as follows: according to the characteristic of the severely paralyzed patient,a face movement recognition method is proposed,which uses Uniform LBP texture feature;based on the coarse-to-fine strategy,an expression recognition method is put forward.The experiment results indicate that the system can identify the human's facial expression,head and face movements accurately.Combining the research content of other units,the next step is to debug and improve the system on the corresponding hardware system.