A Compact High-speed 3D-tracking Module Based On A Compound-eye Camera

The bionic compound-eye vision system, which mimics the curved multi-view structure of the compound eyes found in biology, shows a great potential in expanding the field of view and depth of field of vision system, obtaining three dimensional(3D) information, performing real-time tracking and miniaturizing vision system, thus having gained many researchers’ attention both nationally and internationally. In this study, we propose applying the bionic compound-eye vision system to panoramic high-speed 3D-tracking, and design a compact compound-eye system with a size of 60×60×50 mm3. The parameters are investigated for the system structure after analyzing the blind areas and the detection resolution of 3D-tracking. The image quality is evaluated by analyzing the diameter of the defocused spots and the distortion map.This system could acquire 3D information based on the principle of binocular vision. The imaging model and distortion model of the compound-eye system have been deduced, and the compound-eye lens has been calibrated simultaneously using Tsai’s two-step method. A model has been set up to analyze the 3D detecting precision, which shows the affect on the 3D detecting precision of the structure parameters and the detecting distance.To obtain 3D information of fast-moving objects, a high-speed 3D tracking strategy has been proposed. A camera with dynamic regions of interest(ROI) is used to resolve the conflicts among wide field of vision, high spatial resolution and real-time characteristic by reducing the image area that needs to be exposed, transmitted and processed. To make sure that the image of the fast-moving object falls into the small exposed window, Kalman filters are used to predict the moment of the object images. The parameters of the Kalman filters used in the compound-eye system, such as the state equation, the measuring equation, the initial values of the states and their covariances, have been calculated.The field of view of the compound-eye system is divided by the ommatidia, so that the moving object would be imaged by different ommatidia. Therefore, the ommatidia that have imaged the object must be identified to adjust the 3D detecting parameters and the position of the ROI correspondingly. A method to identify the imaging ommatidia is proposed based on the unique structure of the compound-eye system. The time cost of the algorithm has been analyzed, which indicates that the minimum time cost for per frame is 300μs when the ROI is 40×40(pix×pix) in size, which means that the fastest tracking speed is 3000 frame per second. Inspired by the monolithic multi-view structure of the compound-eye system, a multi-base stereo matching algorithm based on epipolar constraint is proposed to tracking multiple objects.Experiments have been conducted to validate the performance of this system. The experiments test the 3D detecting error of the system, and evaluate the effect of image grabbing speed, the speed of the moving object and the detecting distance, as well as the performance of multi-object tracking. The result shows that: the absolute error of the measured coordinate along the optical axis was 11 mm, while that in the plane perpendicular to the optical axis was 0.4mm; improvement of frame frequency would increase sampling density, which helps eliminate blind areas of reconstruction and discover details in object’s motion; there is an incremental effect on the detecting error of the distance between the object and the optimal detecting position, and the effect is smaller when the object is moving towards the camera; when two objects are tracked, the one that is closer to the optical axis of the center ommatidium has a lower detecting precision. Factors affecting the detecting precision are analyzed, and effective measures to improve the accuracy are discussed, which may lay a foundation for the future design of panoramic high-speed 3D-tracking compound-eye system.