Eye Location And State Classification Based On CNN

As an important part of face,eyes can reflect many significant information.Eye location and state classification(open/close)is widely used in computer vision,such as face detection,expression recognition,pose estimation,human-computer interaction and so on.In addition,the blinking frequency can be calculated and the fatigue state can be judged by eye location and state classification,which is more convenient and faster than wearable devices using ECG or EEG.As traditional methods of eye location and state classification are easily affected by facial expression,pose,illumination,occlusion,background interference and image quality,it is great significant to find an accurate,real-time and robust method.With the development of deep learning,Convolutional Neural Networks(CNN)has been widely used in computer vision.Compared with traditional methods,convolution neural network can automatically extract image features,and has good performance in object detection.Therefore,aiming at the problems in the traditional algorithm,this paper will locate and classify the human eyes based on convolutional neural network.The main works of this paper are as follows:(1)A convolution neural network is trained to detect the eye center point.The network can quickly detect the eye center point from the facial image,and get the specific coordinate values.With the coordinate of the eye center point,the eye area can be determined and the eye images can be obtained.Then,the eye images are input into a classification network,which can determine whether the eye is open or close.Experimental results show that the algorithm is more robust and real-time,and has a good application prospects.(2)Based on Single Shot MultiBox Detector(SSD)network,an experimental data set is constructed.The data set not only contains the coordinate positions of face,eyes,nose and mouth,but also provides the opening and closing labels of eyes.This data set is used to train a SSD network,which can detect human faces,locate human eyes and classify the eye states.When an image is input,this network can detect the eye area and state directly,and when the eye target is too small in the input image,this network also can detect the face at first,and then detect the eye area and states on the face image.In addition,this method is not limited by face detector and robust to the pose change.(3)The proposed algorithms are applied to fatigue detection.Experimental results show that the method of eye location and state classification based on convolutional neural network can achieve accurate and real-time fatigue detection.In addition,the hardware requirement of this method is low,so it can be applied in real applications.