Modeling And Identification Of Eye And Limb Interaction Movement Based On Hybrid Ensemble Learning

With the rapid development of the human-computer interaction technology and virtual reality (VR) technology, the boundary between real world and virtual world has been further blurred, and more and more computer systems adopt the "personification" feature. The "personified" computer system can "sense" the real world by the aid of various sensors, and respond to the simple somatosensory interaction, which will improve user’s experience to some extent. Due to the rapid development of Artificial Intelligence Technology, the "personified" computer system has become smarter and more accurate. It can even identify and make response to the human body movements by observation and learning, thus creates a better immersive virtual reality environment. The eye and limb movements represent almost all the human body movements, and their coordinated interaction can help most people realize the attention selection, therefore, the research on human eye and limb interaction movements is of great importance.In the paper, the main research subject is the modeling and identification of eye and limb interaction movements. The paper will unfold from the following four aspects. 1) How to meet the real-time performance and accuracy requirement of the human-computer interaction system in the virtual reality.2) How to extract the features of the two dimensional eye movements and realize the rapid modeling.3) How to extract the features of the three dimensional limb movements and realize the rapid modeling.4) How to build a unified machine learning model that can process the 2D and 3D movements concurrently.The main contributions and innovations of the paper are presented as follows:(1) The Hybrid Ensemble Learning Model is proposed for the learning and identification of eye and limb concurrent interaction movement. Combined with the Error Back-Propagation Model, Incremental Extreme Learning Machine Model and Ensemble Learning Model, the new Model can learn and identify different types of interaction movements by the corresponding sub-networks. The new model proposes a unified learning model for the identification of eye and limb concurrent interaction movement, which has overcome the disadvantages of the other method, such as the slow training speed, unstable model, and complex network structure, etc. In theory, the new model can minimize the output error of the network close to zero with fewer hidden layer nodes and by multiple auto-increments. The experiments indicate that by comparing with the other machine learning models, the new model can achieve a higher classification accuracy, faster learning process and more stable learning effect with fewer hidden layer nodes and research samples, which tends to meet the requirement of real-time performance and accuracy of the learning and identification of eye and limb interaction movement.(2) The modeling and identification method of eye movement based on Hybrid Ensemble Learning is proposed. Combined with the image topology analysis, Haar feature model and Hybrid Ensemble Learning, the method overcomes the limitations of the other methods when applied in the environment of natural human-computer interaction. Taken the intelligent detection of the drivers’fatigue driving as an example, the comparative experiments verify that the new model proposed show a better robustness, a higher identification rate and a shorter learning process when we learn and identify the movement of human eye in an unfavorable condition.(3) The modeling and identification method of human limb movement based on Hybrid Ensemble Learning is proposed. The new method is consisted of three parts:1) The modeling of human limb movement based on the 3D motion history image.2) The description of human limb movement based on the 3D Hu invariant moments.3) The learning and identification of human limb movement based on the Hybrid Ensemble Learning. Taken the standard traffic-guidance gestures as the samples of limb movement identification, the comparative experiments under different experimental conditions indicate that the new method has a higher adaptation to the application requirement of the natural human-computer interaction in the virtual reality environment than the traditional methods, such as BP algorithm, SVM, and DBN.At last, the modeling and identification technology of eye and limb interaction movement has been applied to the intelligent display system of traffic information. The system can solve the problems in eye and limb movement identification, such as complex data structure, multiple batches learning, and high fuzziness etc. It also provides a unified machine learning application for the learning and identification of eye and limb movements. In the traffic information display system, user can use eye and limb interaction movements to fulfill the task of users’ attention selection.