Research Of Mental Fatigue Recognition Based On Extenics And Facial Visual Cues

With the increasing proportion of mental work in the human production and life, the negative effects arising from mental fatigue are growing, and involved inalmost every aspects ofsocio-economic, military, and daily life. Development of information technology makes it possible to perform real-time mental fatigue recognition. Multi-feature fusion non-intrusive recognition of driver fatigue based on facial visual cues and computer vision technology has achieved some results.However, facial visual cues based mental fatigue recognition faces two major challenges. The first one is thatmore effective mental fatigue facial visual cues should be studied, and the accuracy of the feature extraction must be improved. The second one is that we should find the way to formalizing contradiction problems existed in multi-feature fusion process, establish the formal solution model of contradiction problems, so we can use computer to automaticallyand intelligently find the optimal strategies and methods for multi-feature fusion.This dissertationfocuses on the above issues to study, and the main research work are as follows:1. Research of mental fatigue recognition extension model based on facial visual cuesThe concept of human mental fatigue was defined to identifytheresearch target of this dissertation. Theincompatible problems existed in facial visual cues based mental fatigue recognition were analyzed, the general method to generate extension strategy for solving contradictions problems was introduced. Accordingly, this dissertation established the extension model for mental fatigue recognition, and the basic-element and extension model, extension strategy for the basic-element were studied.2. Research of dynamic facial features based mental fatigue recognition For the existing facial visual feature extraction methodscannot effectively deal with the large angle of head rotation and tilt, as well as hair cover, this dissertation proposed apractical iris segmentationand blink parameters calculation method, and a mouth inner contour corner point curve fitting based mouth open degree calculation method.For the blink parameters threshold based mental fatigue recognition methods are susceptible to blink habits and age, and have poor adaptability, this dissertation proposed a mental fatigue recognition method base on theFRdigitalfeatures of the blink parameters according to the statistical features of blink parameters for different age groups. For the mouth open degree thresholdbased mental fatigue recognition methods are susceptible to size of the mouth and thickness of thelips, this dissertation proposed a yawning detection method based on dual-threshold of mouth open degree and mouth open duration.For the recognition rate of mentalfatigue transition stage is not high based on theFR digitalfeatures of the blink parameters, this dissertation established a BD-time series based on blink duration,and proposed a segmentation method and a HMM based classification method for it. For the dual-threshold based yawning detection cannot distinguish deep yawn and light yawn, this dissertation established a M-time series based on mouth open degree and duration, and proposed a segmentation and classification method for it. Our methods improve the recognition accuracy of the transition stage and can distinguish yawning of different mental fatigue degree.3. Research and recognition of the chronic facial fatiguefeatures of mental fagtigueAfter a long time observation of the facial videos from the Sleep Disordered Chronic Fatigue Syndrome(SD_CFS) patients, whose CFS are mainly caused by work and life stress, we found that the SD_CFS patients have the chronic facial fatigue appearance, such as looking bleak, the sleep silkworm is prominent, frowning, and curl the lips. So, this dissertation proposed the feature extraction,feature fusion,feature selection and classification method for the chronic facial fatigue appearance,therefore provides an feasible way to non-intrusively estimate work and life stress.4. Mental fatigue recognition based on hierarchical extension fusion of multi-featuresand multi-methodsThis dissertation proposed a mental fatigue recognition method based on hierarchical extension fusion of multi-features and multi-methods after analyzingthe advantages and disadvantages of the mental fatigue features and recognition methods proposed above. From the single feature layer, dual-feature layer to multi-feature layer, this dissertation dissertate the ideal and mechanism of extension fusion, the definition of mode, mode switching rules, contradiction judge method in each layer hierarchically.This dissertation finally summarized its main research results and innovations, pointed out the future research directions.