Research On Compound Optical Bionic Vision Key Technology

Eye is the most important sensory organ for human to perceive external environment information.For robot system,vision sensor is also an important perception device by imitating human eye functions.It can help robot to sense environmental information quickly and make sound decisions.At present,among many research methods of robotic vision system,the bionic technology by simulating biological adaptive mechanism has been paid much attention,and it is of great practical significance to research and simulate the multimodal visual signal collection,transmission and processing process of human visual system,and then apply these research results to the robot vision areas.Therefore,this dissertation focuses on simulating the multimodal transmission and processing mechanism of human eye,and then several key issues of bionic vision such as perception,reconstruction,target detection,tracking and face recognition of intelligent robot in complex dynamic environment can be solved by analyzing and processing the heterogeneous sensors image effectively.Finally,the compound optical bionic vision system of intelligent robot is constructed based on the above research.the main research contents of this dissertation are as follow:(1)According to the difference of face representation information reflected by visible and thermal infrared multimodal images,a multi-spectral face image registration method based on student's-T distribution mixture model is proposed.The method solved the multi-spectral face image registration problem by taking full account of both the feature point sets spatial global structure and local shape feature.On the one hand,we use inner-distance shape context as the local shape feature of feature point sets,and create the initial matching relations of multi-spectral face image.On the other hand,the exact matching relations and spatial transformation model of the multi-spectral face image can be parameterized by a student's-T mixture probabilistic model,and the parametric model can be solved by using expectation maximization algorithm.Furthermore,in order to improve the anti-interference performance of follow-up face recognition method,a guided filtering and gradient preserving image fusion scheme is used to fuse the registered multi-spectral face image,which can make the multi-spectral fusion image hold more apparent details of visible image and thermal radiation information of infrared image.Subjective and objective comparison experiments of different registration methods with UTK-IRIS standard multi-spectral face database and self-built multi-spectral face datasets demonstrate the robustness and efficiency of our proposed registration method in solving multi-spectral face image registration problem.(2)In order to get rid of the dependence of active hand-eye calibration method on accurate target,by combining with monocular stereo vision——structure from motion algorithm,a new extended model of passive hand-eye calibration is proposed in this dissertation.and the optimization methods of hand-eye calibration extended model based on second-order cone programming and bundle adjustment are proposed respectively.The latter optimization method based on bundle adjustment can solve simultaneously the optimal solution of robot-world and hand-eye calibration as well as 3D object reconstruction.A large number of simulation and measurement experiments based on general scene show that the hand-eye calibration task can be achieved with calibration target or free of calibration target by the former optimization method based on second-order cone programming,which can break away from the dependence on target and expand the application scope of hand-eye calibration method into general scene.Furthermore,in order to achieve quantitative accuracy evaluation of optimization method based on bundle adjustment,the calibration field is constructed by using photogrammetric retro-reflective targets and reference bars.The accuracy and validity of the method are verified through comparing difference values before and after the calculation of camera position and orientation matrix,as well as comparing the difference results between the reconstruction of feature points and nominal values of reference bar.Calibration results show that the rotation matrix relative error of camera position and orientation is less than 5/10000,the translation relative error of camera position and orientation is less than 8/10000,and the distance measurement mean error about the both ends of 1 m reference bars after 3D reconstruction is almost 0.1mm.(3)The problem of pedestrian detection and tracking in complicated and changeable dynamic environment is researched and analyzed.By taking advantage of object and scene feature complementarity information reflected by visible and thermal infrared multimodal images,a pedestrian detection and tracking method based on multi-spectral compound images is proposed in this dissertation.There are three main points in this solution:Firstly,the infrared and visible dual-field multi-spectral images acquisition system is designed,and a flexible liquid zoom len is used to achieve autofocus function of visible image.Then,in order to enhance the robustness of kernel correlation filtering object tracking method in complex situations of out of sight,partial occlusion,and similar target jamming,the peak-to-side lobe ratio is introduced as an adaptive criterion for detecting whether the tracking result is success or failure.Finally,by imitating primate head-eye coordinated motion mechanism,the control strategy for multi-spectral compound bionic vision system is established.The experimental results show that the proposed pedestrian detection and tracking method can realize accurate detecting and real-time tracking for pedestrian in complex dynamic scenes,and compared with the method based on visible or infrared single spectrum image,the success rate of pedestrian tracking based on multi-spectral images is increased by 22.3%and 7.8%respectively.(4)According to the mission needs of target detection and tracking in complex dynamic environment,a two-stage 4-DOF bionic vision eye-neck motion platform is developed,and a compound optical bionic vision system for intelligent robot is constructed.Furthermore,relying on this bionic vision system,the face image registration,pedestrian detection and tracking experiment is designed based on visible and infrared heterologous imaging sensors.The experimental results show that the proposed image registration,pedestrian detection and tracking methods in complex dynamic environment are accurate and effective.