Research On Multi-camera Multi-dimensional Monitoring System At Intersection

With the construction of smart city and the advancement of Xueliang project,multi-camera video monitoring system is widely used in road traffic management,public security,infras-tructure construction,emergency command and other fields.In particular,for the multi-camera system deployed at the intersection,the number of imaging sensors is increasing,while the monitoring topology is also diversified.In order to make full use of the massive scattered visual information collected by these cameras,the research on multi-dimensional stereo video monitoring methods,breaking the information island between cameras,and re-alizing interconnection has gradually become one of the development trends of related fields in the future.The multi-camera and multi-dimensional stereo monitoring method studied in this thesis has important theoretical research significance and application value in camera space calibration,virtual reality,cross camera analysis and so on.This thesis introduces the research status of related technologies in surveillance system in detail,analyzes the relationship between multi-camera visual data,and focuses on three key issues: multi-camera data integration,multi-camera spatial calibration,video virtual reality fusion and rendering Pose calibration method and multi-channel video virtual reality fusion and rendering,and build a set of intersection multi-camera multi-dimensional stereo moni-toring system.The main research work of this thesis is as follows:(1)A multi-camera multi-dimensional stereo monitoring system is built in this thesis.The system mainly includes three modules: one is multi-camera monitoring data acquisition and coding,which is used to integrate multi-channel scattered video stream data and push it to the server; the other is multi-dimensional stereo scene construction,which is used to combine the virtual model of monitoring scene and 3D geographic information platform; the third is multi-channel real-time video and scene model virtual real fusion,which is used to pull the monitoring data from the server Finally,multi-camera multi-dimensional stereo monitoring is realized.(2)In this thesis,a high point aided multi-camera spatial alignment method based on spa-tiotemporal feature map is proposed to estimate the spatial position relationship of cameras in surveillance system.This method relies on the global perception ability of high point surveillance camera,and achieves camera spatial alignment by mining the consistency of cross view motion information.The proposed algorithm mainly includes two parts: spa-tiotemporal feature map construction and cross view spatiotemporal matching.According to the corresponding relationship of pixels obtained by the algorithm,all ground surveillance cameras are aligned to a unified imaging space.The effectiveness of the algorithm is verified by the experimental evaluation in simulation and real environment.(3)In this thesis,a multi-camera pose calibration method based on SFM is studied to op-timize the relative pose of sensors in multi-camera video surveillance system.Based on the existing SFM method,the algorithm deeply integrates the feature extraction and matching method of deep learning,and establishes the spatial relationship between different cameras by reconstructing the 3D point cloud of the monitoring scene.At the same time,according to the pose consistency between video frames,this method uses the strategy of stable scene fea-ture complementation and interference motion feature filtering,and proposes a multi-camera pose estimation optimization method based on video sequence.Through a large number of experimental evaluation,the algorithm can effectively improve the accuracy of multi-camera pose estimation.(4)This thesis studies the virtual reality fusion of multi-video and three-dimensional scene,which is used to put surveillance video into virtual three-dimensional scene to achieve multi-dimensional three-dimensional video monitoring effect.The research of this part mainly in-cludes camera viewability analysis and video texture mapping.Firstly,this thesis analyzes the visual field of video by improving shadowmap algorithm.On this basis,the image tex-ture is extended to video stream data,and the texture mapping of real-time video stream is carried out on the virtual model.Finally,the virtual reality fusion of multi-channel video and 3D scene model is realized.