The Research On Camera Array Based Occluded Object Imaging And Tracking

The continuous improvement of vision sensors and processors has laid the foundation for the research of camera arrays on the hardware.Unlike a single camera acquisition device,the camera array has a wider field of vision and can acquire more abundant information.As one of the common light field collection equipment,the camera array not only collects the color information of the imaging plane,but also records the angle information of the imaging light.Therefore,the algorithm research based on the camera array breaks the limitation of the traditional single-view algorithm and integrates the complementary information obtained from multiple view,which provides a new solution to the problem of target occlusion in complex scenes.Based on the camera array,this thesis has conducted in-depth research on the imaging and tracking of occluded objects.The research content mainly includes the following aspects:(1)A synchronous acquisition system based on camera array is built,and the calibration method of camera array based on parallel parallax and bundle adjustment is studied.In this thesis,8 network cameras are used to construct a camera array,and a multi-threaded acquisition architecture is designed to ensure the synchronization of data collection in light field.In addition,the principles of camera array calibration algorithms based on parallel parallax and bundle adjustment are deeply studied in this thesis,and their calibration accuracy and imaging results are analyzed on the calibration data.(2)A synthetic aperture imaging algorithm based on CUDA is implemented.The synthetic aperture imaging algorithm based on camera array needs to process rich information and large amount of data.In order to improve the real-time performance of the algorithm and meet the needs of practical applications,this thesis implements the accelerated design of the algorithm based on CUDA by using the image calibration method of the lookup table,the multi-thread parallel parallax elimination and image synthesis strategies.In order to verify the effectiveness of this method,this thesis is carried out statistical experiment verification and analysis of imaging results.The experimental results show that the method has a20.94-fold improvement in operating efficiency compared with traditional synthetic aperture imaging algorithms,and can achieve real-time imaging of occlusion objects.(3)A synthetic aperture imaging algorithm based on semantic information driven is proposed.The algorithm mainly consists of two parts,which are the generation of target depth labeling maps based on semantic information and the data-driven variable aperture synthetic imaging.The first part is the method of generating target depth labeling map based on semantic feedback.Firstly,a deep learning algorithm is used to mine target information on the synthetic image,and this information and target depth are fed back to each view of the camera array to obtain the corresponding target depth labeling map.The second part is the variable synthetic aperture imaging of the specified depth based on the target depth labeling maps.The variable aperture includes two aspects: one is the adaptive variable imaging aperture driven by the target depth and viewing angle;the other is the light filtering driven by the visual information in the target depth labeling map.On the CUDA platform,the occluded target imaging after occlusion information eliminated is achieved.Evaluations on several complex indoor scenes and real outdoor environments demonstrate the superiority and robustness of the proposed approach.(4)A camera array collaborative tracking algorithm based on the association of spacetiming.Due to the spatiotemporal consistency of the target in the scene,the complete track of multi-target tracking can be obtained by the temporal and spatial data association of the target at any time.First,a human head detection model is established based on the YOLOv3 network.In order to effectively solve the detection problems caused by pedestrian occlusion and view changes in the scene,this thesis constructs human head data sets containing different distances and different angles to train the network.Then,timing and spatial data association are carried out for the target on the basis of detection.In terms of timing,the cost matrix is constructed based on the similarity of the target overlap model,so as to correlate the target and obtain the moving track of the target under a single camera.In terms of space,a target similarity model based on epipolar geometric constraints is designed to complete cross-view data association.After that,the single-camera tracking results are corrected according to the spatial correlation information of the camera array,and finally a continuously stable multi-target tracking trajectory is obtained.Finally,the algorithm is tested on a self-built camera array multi-target tracking data set,which verifies the effectiveness and robustness of the algorithm.