Fatigue Recognition Based On Blink Detection Using Adaptive Threshold

Fatigue recognition can be applied to fatigue driving warning,air traffic controller fatigue monitoring,giant machine operator fatigue warning etc,in order to avoid the huge security risks.In view of the lack of considering the individual differences and dependence on laboratory data of the existing methods,the work investigates fatigue recognition in reality scenes based on blink detection using adaptive threshold and normalized individual difference processing.Effective feature extraction is the premise of reliable and effective fatigue recognition.Blink detection is the key technology to extract eye features,but the existing methods less consider the individual differences,resulting in the lack of generalization ability.To overcome the shortcoming,the work proposes an adaptive eye aspect ratio threshold method based on Kalman filter and waveform features threshold,which can adaptively capture the blink waveform without the limitation of feature change in individual and periods.Furthermore,the proposed method can realize real-time blink detection and blink duration estimation simultaneously.The proposed method is verified on three datasets and shows high effectiveness,which can ensure the effective fatigue features extraction.Based on the eye features extracted by blink detection using adaptive threshold,the work utilizes machine learning to build the fatigue recognition models.On the one hand,the existing methods designed by laboratory data cannot be suitable for realistic scenes;On the other hand,individual differences in fatigue bring challenges to effective recognition.To overcome the first limitation,the work establishes the fatigue recognition model by using 99 videos of 33 subjects recorded in realistic scenes.In order to reduce the measurement error caused by blink detection,a sliding window averaging method combined with blink detection using adaptive threshold is proposed to extract features.To overcome the second limitation,this paper proposes a normalization method to process features.The obtained features are compared with the standardized features and the unprocessed features,and the results show that normalization method enables the features to be more consistent.Based on the features obtained by different processing methods,the work investigates the fatigue recognition models implemented by four classifiers:support vector machine,multilayer perceptron,decision tree and extreme gradient boosting tree.The results show that considering individual differences can greatly improve the performance,and the normalized features paired with the extreme gradient boosting tree model(normalized model)achieves the optimal accuracy of 73.3% and 97.5% in the three classification task and binary classification task of fatigue degree,respectively.In addition,a variety of metrics are selected to compare the effect of the proposed normalized method with the existing standardized method,and analyze the comprehensive performance of the normalized model.The results show that the performance of the normalized model is better than that of the standardized model and the model without considering individual differences,which proves the effectiveness of the constructed model.Finally,the proposed model is compared with the HM-LSTM model based on homologous data.The results show that the normalized model has the advantages in feature extraction technology without training,less input parameters and better interpretability.