The Study Of Registration And Fusion Method Of PTZ Camera Video In Augmented Virtual Environments

PTZ(Pan-Tilt-Zoom)cameras are widely used in security.General video surveillance systems are built by multiple independent video monitors.Because the videos on these monitors are scattered and unorganized,the monitoring personnel might make mistakes because of the incoherence and incomplete information.Augmented virtual environment technology can make 2D video embedded in 3D scene so that the virtual scene can be combined with real world.With the combination of abstract things and realistic world,we can be better to understand spatial relationship of videos and scenes and we can get more comprehensive spatio-temporal information and broader view to promote security service.Camera registration in the 3D scene is the premise of the fusion of the video image and the virtual 3D scene.The common method of camera registration is using artificial markers placed in the scene.This method is hard to use when the camera is being dynamic adjusted.Nowadays,most surveillance cameras are PTZ cameras and they can't offer their camera exact parameters in time.It makes the fusion of videos and virtual scenes infeasible.In this paper,based on computer vision theory and technology,we studied the method for the fusion of PTZ camera videos and 3D scenes.The main research contents and results are as follows:(1)We analyzed the relationship between the real camera and the virtual camera in the 3D scene,and cleared the relationship between the camera internal and external parameters of real camera and settings parameters of 3D virtual camera.With the combination of linear and nonlinear methods and point coordinates from the high precision 3D models,we can get the parameters of real camera and finish the camera registration without artificial markers.(2)We made classified discussion about the real-time changes of PTZ cameras and put forward a method for real-time camera registration based on computer vision theory.Firstly,we use the global characteristics of the video frames to rapidly detect whether the camera parameters changes and then use the local feature precise matching to camera registration and finally get the video combined with virtual scene automatically.This method considers the efficiency and quality of fusion work and avoids the complexity and hysteresis of the manual adjustment.(3)After camera registration step,we analyzed the principle how the virtual camera projects video texture.By using the OpensSceneGraph 3D rendering engine,we achieved the fusion of PTZ camera real-time videos and virtual 3D based on the ShadowMap algorithm and GPU shader technology.Experiments in this paper prove the validity and reliability of the proposed method to a certain extent.The research on real-time calibration of PTZ camera can also be extended to the real-time calibration of mobile camera video,which is helpful to expand the practical application of the virtual environment system.