Head-eye Dual-channel Intelligent Human-computer Interaction System

Nowadays,the more intelligent and natural multi-channel human-computer interaction technology(MHCI)is becoming the development direction of human-computer interaction technology.Visual interaction occupies an important position in MHCI due to its intuitiveness and other advantages,and eye tracking technology,as the technical support of visual interaction,has become one of the hot research directions in recent years.Eye tracking technology has developed to the present,but there are still problems such as limited head movement and difficulty in achieving both accuracy and real-time performance,resulting in poor practicality of the technology.Head pose,as another key technology in MHCI,has limited application scenarios due to its fuzzy directional interaction characteristics.However,the most intuitive interaction intentions can be obtained through head-eye behavior.If the head movement and eye movement can be used as dual-channel input and their respective advantages can be palyed,the naturalness of human-computer interaction can be greatly improved,and at the same time,it also provides a good solution to the problem of human-computer interaction for the disabled.Therefore,this thesis designs a head-eye dual-channel intelligent human-computer interaction system based on eye tracking technology and head posture detection technology.In terms of the system hardware architecture,this thesis uses the head-mounted device to carry the whole system hardware circuit,and based on CMOS image sensors and MEMS motion sensors,with field programmable gate array(FPGA)as the core to increase the flexibility of the system design,and at the same time use the advantage of parallel processing to accelerate the algorithm,thereby improving the real-time performance of the system.In the design of the system algorithm,according to the characteristics of the human eye images under the near-eye camera,a coarse and precise eye tracking algorithm is proposed.The image of the pupil region is obtained through the region of interest,then using threshold segmentation,morphological processing and centroid method locate the pupil center step by step,at the same time,according to the characteristics of the output data of the gyroscope and accelerometer in the motion sensor,the complementary filtering algorithm is adopted to realize the calculation of the head posture.Inside the FPGA,the hierarchical idea is used to divide the functional modules,and Verilog is used to describe the logic.Through the cooperative work of internal and external circuits,the collection of human eye images,the processing of eye tracking algorithms,the collection and calculation of head posture data,and the data transmission with the host computer are realized.The system test results show that the average accuracy of the pupil center positioning and head posture detection of this system is more than 95%,and it can process the human eye images and head posture data in real time.At the same time,the head and eyes dual-channel coordination can be realized for human-computer interaction by combining with the host computer driver software.