Video Data Analysis Of Post-stroke Depression

Depression is a serious mental illness that harms human health, with high incidence, high recurrence, high suicide rates, high morbidity and heavy burden on society, etc.However, most symptoms of depression don’t catch the attention of patients, families and doctors.Partly because depression have complex symptoms, in addition to psychiatric symptoms, there are often biological or physical symptoms at the same time; on the other hand, doctors often lack systematic training of depression treatment technology, who usually just treat the physical symptoms of the patient and its related diseases, leading to the delay treatment. Facial expression, eye gaze and head movement provide important visual clues to estimate degree of depression, this paper aims to predict patients’degree of depression with video information. We developed a video recording program and capture patients’videos.In this paper, we applied data analysis to the interview videos of post-stroke depression with doctors to study the correlation of multi-dimensional expression features.After which, we can identify typical depression expression feature and establish the prediction model of depression. Main work of video data analysis included preprocessing, single-frame AU motion detection and probability estimation, video sequence analysis and video feature extraction, mining depression expression feature and establishment of depression level prediction model.In the single-frame AU detection section, we not only trained 13 kinds of AU classifiers, compared to the previous study, covered more AU, that can be helpful of more comprehensive and in-depth research on depression facial movement patterns; We also apply multi-classification probability estimation method to estimate the probability of the emergence AU, compared to the simple binary classification result, it is possible to characterize the strength of the AU finer movement. Probability sequence is the foundation of video feature extraction and analysis.In the feature extraction section, we not only extract the single-frame feature, but also extracts video feature with HMM model. HMM algorithm model parameters of each AU were learnt by EM algorithm, based on which we estimate the AU’s hidden states.With consideration of the relationship between video frame before and after, we get higher accuracy than the single-frame AU detection. Sequence of AU states were extracted as feature vector of each video.In the feature mining section of depression videos, we first studied the correlation between depression and single AU, and get consist conclusion with the Affective Dsyregulation hypothesis. With the severity of depression, behavior with negative pleasure will increase while behavior with positive pleasure will decrease; We then research typical depressive expression named smile control, and it was found along with the severity of depression, smile control appeared to be more frequent, indicating that patients with depression have a tendency to suppress feelings of pleasure.In the prediction section of depression degree, we proposed two prediction models, regression model and KNN model to predict five levels of severity of depression.And contrast their prediction performance on 51 new depression videos.The result shows that KNN algorithm is more suitable for the problem of depression degree prediction. We got 88% accuracy with limited training data.