Omnidirectional Imaging And Video Target Tracking

With the increasing demand for image information in modern society,omnidirectional imaging technology and its applications have been continuously developed.Target detection and target tracking technologies for omni-directional video have also become a popular multidisciplinary branch in the field of computer vision.These technologies are widely used in technical application scenarios such as inspection robots,aircraft guidance,automatic driving,and security monitoring.This paper focuses on the omni-directional imaging technology based on multi-camera stitching.On this basis,it realizes real-time detection of moving targets,and conducts in-depth research on the cooperative target tracking combined with the PTZ camera.Firstly,for the system design,four fish-eye cameras were chosen to establish an omnidirectional imaging system.For the distortion characteristics of the fish-eye camera,a high-order distortion model is selected,and the internal parameter matrix and the distortion coefficient matrix are optimized by bundle adjustment.The space constraints of the picture are consistent with the human visual sense through the cylindrical projection method.In the image registration stage,a variety of feature extraction methods are analyzed,and the Affine-AKAZE algorithm is proposed by combining the full affine simulation framework and the AKAZE algorithm.The superiority of the Affine-AKAZE algorithm in large parallax scenarios is verified through comparative experiments.After the matching point pairs are obtained,the projection matrix is optimized by bundle adjustment,and the coordinate alignment of the omnidirectional imaging system is realized.A linear fusion algorithm is used to make a field of view image softly transition to the adjacent field of view image.In the panoramic image generation stage,the original image coordinates of the pixels are pushed back step by step according to the transformation relationship,and a fast mapping table is constructed.The rapid mapping method is used to quickly generate a panoramic view and real-time all-round video surveillance is realized.Secondly,the in-depth research is carried out on the target detection algorithm.In terms of traditional algorithms,after studying and comparing several traditional algorithms such as frame difference method and background modeling method,an innovative target detection algorithm based on the improved Vi Be is proposed.By setting an adaptive foreground threshold,the algorithm improves its robustness to scenes with uneven lighting in the background environment.Through the combination of Vi Be algorithm and frame difference method,the ghost image problem is solved.Through morphological methods,the integrity of the detection target is guaranteed.In terms of deep learning,the paper researches the latest YOLOv4 algorithm and clarifies the reasonable and advanced aspects of its framework.Considering the limited hardware conditions of the system,the paper chooses to build a lightweight YOLOv4-tiny network model for target detection,successfully applies it to the system,and evaluates its actual performance and feasibility.Finally,this paper focuses on the video target tracking algorithm and the cooperative tracking combined with the PTZ camera.The Kalman filter algorithm is used to model the target's position,speed and other attributes.Through continuous prediction and update,a more accurate prediction of the target's trajectory is formed.The smoothness of the tracking trajectory of the moving target is ensured,and the influence on the tracking effect when the speed and direction change suddenly is reduced.A cooperative tracking system combined with the PTZ camera is established,and a cooperative tracking automatic registration algorithm is innovatively proposed.This algorithm establishes the spatial mapping between the omnidirectional imaging system and the PTZ camera coordinates,and establishes the corresponding relationship between the target coordinates in the video and the attitude angles of the PTZ camera.A set of control algorithms for the PTZ camera is proposed.The target selection,P/T control,and Zoom control are carefully designed,and methods such as central area threshold method,fuzzy PID control and table look-up method are introduced to realize the real-time tracking of the most threatened target and obtain the clear details of the target.In this paper,a video surveillance system based on omnidirectional imaging combined with the PTZ camera for moving target tracking is studied and realized.It can not only grasp the overall security situation,but also track and monitor the key targets.The combination of the two greatly improves the intelligence of the security monitoring.