Research On Controller Status Detection Based On Deep Learning

Fatigue risk management and control of air traffic controllers is an important part of aviation safety research of civil aviation of China.Once the controller fatigue problem is serious,it may directly or indirectly cause a series of accident symptoms,or even cause accidents.Therefore,it has important research value to detect the fatigue and related actions of controllers by objective methods.At present,for the fatigue detection of controllers,most of the state detection methods are subjective evaluation methods,and less is objective evaluation methods.In the objective evaluation method,computer vision has been applied because of its economy and non-interference.But at present,the fatigue detection of controllers is limited to facial feature information.In this paper,in addition to facial information,behavior and action also contain many information elements.Mining these information is helpful to analyze the fatigue phenomenon and related state of controller.Therefore,this paper introduces action recognition into the field of ATC safety monitoring.At present,the fatigue threshold selection of controller based on face information mostly adopts the standard of vehicle fatigue,which lacks the personalized evaluation index of controller.In order to solve this problem,this paper takes several first-line controllers as the experimental objects,and designs the relevant experiments to get the change rule of relevant indicators applicable to controllers.In this paper,the state detection method combined with multi detection algorithm,through image processing,deep learning technology gradually locates the face,the human eye area,locates the key points of the human skeleton.Two modules are constructed,eye state detection module and limb movement detection module.The face detection module uses mtcnn neural network model to calibrate the human eye area of the controller,and then uses convolutional neural network to determine the open and closed state of the eyes,and calculates the physiological fatigue mature index PERCLOS.The limb movement detection module uses OpenPose and alphapose algorithm to mark the key points of the human skeleton,and uses the key points of the skeleton as theinput to a deep neural network to identify the corresponding actions.Combined with the face fatigue module to determine the fatigue or judge the behavior of the controller separately.Finally,the experiment is carried out on 78 first-line controllers of Southwest Air Traffic Control Bureau of Civil Aviation Administration of China.