Upper Extremity Behavior Research For The Pilot Smart Model

Aviation safety guarantees the effective operation of civil aviation security and directly affects the personal and property safety of the passengers, the airlines reputation, economic benefits, as well as the airline's survival. Flight crew operation error has become the main reason for civil aircraft crash which is mostly led by human factors in civil aircraft accidents. So the hot point of aviation safety has been gradually shifted to the ergonomics combination of human, machine and environment.Cockpit man-machine interface is the most direct and important position for ergonomics. The study of pilot behavior and smart model is the most important and critical work of the ergonomics research. The pilot operation tracking and behavior analysis based on computer vision is the basic technology for the cockpit human factors. Currently, progress has been made in military aircraft on such technology, but still at the initial stage in the civil aircraft.In this paper, we use visual surveillance technology for the operation gesture features research of the pilot smart model. The main contribution include machine vision-based operating gesture detection, tracking, operation trajectory behavior analysis and the pilot driving behavior analysis combining pilot's eyes movement characteristics. Hand detection and tracking is the basement of the trajectory behavior analysis. First, we collect the video and use pretreatment technology to solve the image degradation problem caused by environmental or equipment factors. And then, we detect hands gesture by background subtraction and skin color model, and track the gestures movement by camshift algorithm and kalman filter technique. Additionally, we use the pyramid optical flow to detect gestures local fine-operation action. Last, behavioral analysis in the cockpit environment has been carried out based on the detection and tracking of hand gestures: 1. the gesture trajectory distribution patterns are learned by the improved hierarchical self-organizing neural network, and then the learned patterns are used for gesture trajectory prediction and anomaly detection; 2. the construction of pilot smart model is explored in this paper, as well as the joint analysis of the timing event model based on hand-eye coordination; to provide reasonable data analysis logic for the pilot smart model, the scenario of hand-eye cooperation action has been built on the approach and landing phase. As the result, we have explored a system which can detect, track the pilot's operating gestures. The system can also use the historical data to train the intelligence model. The model can predict the current operating trajectory, and detect the abnormal event. The system can be extended for the hand-eye coordination modeling and analysis. It can be very helpful for the further study of the cockpit ergonomics.