The Multimedia Play Control System Based On Facial Video

Imaging Photoplethysmography(IPPG)can extract the subtle information of dynamic vital signs from videos of body parts such as the face or fingertips.With the popularization of IPPG technology,various kinds of signal processing methods have been used to extract various physiological signals from video,which has become a research focus.The traditional human-computer interaction(HCI)cannot meet human needs,so the convenient,natural and non-contact HCI mode has gradually attracted human’s attention.This thesis focuses on the non-contact implementation of HCI system based on facial actions and physiological signals.For this reason,IPPG technique combined with temporal and spatial filtering methods was studied concerned with the detection of eye-blinking,yawn as well as blood volume pulse(BVP).Aiming at improving the performance of IPPG technique applied to facial information achieving,G component-and independent component analysis(ICA)-based IPPG methods were investigated and compared.To address the inherent limitation of ICA in output ambiguity,spatial and frequency patterns of independent components were employed to recognize BVP and eye-blinking signals respectively.Based on the G component-based IPPG method,a HCI system for multimedia control was designed and implemented.This HCI system can achieve the information of eye-blinking,yawn and BVP in real time and transform it into commands to control multimedia player.The main work of this thesis is as follows:Firstly,this thesis analyzed the research status of three physiological:eye-blinking,heart rate and yawn,and then analyzed and compared two face detection algorithms,which based on Viola-Jones and based on skin-color,respectively.Finally,we chose the skin-color-based face detection algorithm.Second,in this thesis,two physiological characteristics detection methods:G component-and ICA-based IPPG methods were introduced respectively.And proposed that the BVP signals and eye-blinking signals were identified according to spatial distribution feature of the mixing matrix and frequency domain features respectively.During detect the yawn,according to the mouth open duration,we effectively distinguish astonishment and yawn.Thirdly,we compared the eye-blinking detection performance of G component-based IPPG method and ICA-based IPPG method in four scenarios:with light change,without light change,with glasses and with head movement in different directions.The influences of IPPG signal extracted from different areas and interference signals such as light on heart rate detection were analyzed.The differences between astonishment and yawning were compared,and the problems of various disturbing signals on yawn detection were analyzed,too.Quantitative experimental results revealed that the G component-based algorithm achieved a lower complexity and a faster speed but sensitive to the selected areas of different physiological characteristics.The ICA-based IPPG method has obvious advantages when there is light or movement interference,and can separate different physiological characteristic signals based on the signals extracted from the same area,however,the complexity and speed of the algorithm are unsatisfactory,which are not conducive to the development of real-time systems.Fourthly,in order to verify the feasibility of continuous eye-blinking in multimedia play control,the characteristics of eye-blinking pulses(such as effective eye-blinking in unit time and pulse morphology)for different subjects were analyzed according to the proposed method of continuous eye-blinking detection method.On this basis,the parameters for time/space analysis window and filter were selected and adjusted,for eye-blinking pulses detection and related control command generation.We developed the multimedia play control system based on facial video in the C++ environment,and the detection of eye-blinking,heart rate and yawn were implemented.The experimental results shown that,users can effectively control the multimedia player by using the developed HCI platform with the accuracy of 92.95%using the continuous eye-blinking detection method proposed in this thesis.The proposed eye-blinking detection algorithm and eye-blinking control based HCI system have good potential for application and operation promotion.