Research On Eyeblink Detection In The Wild

Eyeblink,which is a typical action of the human body,indicates current physical and psychological state of the individual.Recently,people have gradually discovered that eyeblink detection can be used with a wide range application in real-life,such as behavioral living confirmation of living detection,recovery of exophthalmia in the medical field,lie detection in criminal investigation,and fatigue detection in assistance driving.Therefore,a large amount of works towards eyeblink detection have been carried out in the past decades,and relatively satisfactory results have been achieved.However,most of the current methods focused on blink detection towards a specific constrained dataset,which is far from the conditions in the wild.Therefore,how to detect eyeblink in the wild will be an important and urgent research direction.This paper reveals that compared with the eyeblink in a specific constrained dataset,the eyeblink in the wild has bigger changes in four aspects: human attributes,human posture,illumination,shooting angle and distance.Based on these discovery,analysis shows that eyeblink in the wild has four characteristics: unrestricted target,unrestricted target's posture,unrestricted environment,and unrestricted shooting.Therefore,the film source which has similar characteristics is used as the sample source to collect and establish a wild eyeblink,named HUST-LEBW.So,blink characteristics in the wild are visually described.With the performance evaluation of some state-of-the-arts in this wild eyeblink dataset,challenges of blink detection behavior under unrestricted conditions are illustrated.Based on the above explorations,this paper proposes a technique for eyeblink detection in the wild.First,a face parsing algorithm is used to locate the eye region,and then a tracking algorithm is used to track subsequent eye regions.After that,artificial descriptor features are extracted from the obtained blink areas,and the temporal action information is captured using the long-short time memory model to perform eyeblink verification.However,as duration of eye-blink is different with each other,using Long Short-Term Memory models directly may have limited performance.Therefore,this paper proposes a multiple time scale Long Short-term memory model.Specifically,using multi-layer Long Short-Term Memory model to mine multiple time scale information;at the same time,we propose a multiple time scale feature to describe the captured multiple time scale information.Further,through the above research,it has been found that the eyeblink detection model in the wild needs to locate the eye region robustly,and to extract sufficiently discriminative features from it.In view of this,this paper proposes a method for mining robust and discriminative eye region.Specifically,on the premise of ensuring eye localization's accuracy(i.e.,robustness),a potential eye area is located.Then jointly optimize the two modules,whose are eye region localization and blink verification through reinforcement learning to obtain a discriminative eye region.At the same time,consider overfitting problem in reinforcement learning,a new reward function is proposed to balance the problems of overfitting and underfitting.Finally,considering the multiple time scale natural of eyeblink itself,and in the view of multiple instance examples,a weighted fusion model based on VLAD coding is proposed to further improve the performance of the blink detection model.