Spatio-temporal Correlation Access Of Multi-channel Video And Visual Information Fusion

The acquisition of video and navigation information is widely used in the fields of unmanned vehicle,robot rescue,agricultural survey,military command and so on.The video has the characteristics of time series,and the navigation information has the characteristics of space,geographical position,posture,movement and so on;Different from the common sensor data,the video's frame has a large amount of data,complex format and different acquisition frequency,it's transmission mechanism is complex and requires strict delay and network bandwidth,the association access of multi sensor is a research focus.Video mosaic and video super resolution reconstruction are 2 important techniques in video information fusion,which can be used in video surveillance,military command,virtual reality,3D and other fields.Different from a single camera,a plurality of video cameras with overlapping regions have redundancy and local similarity.Multi camera video mosaic can obtain multi pose,large viewing angle,no blind area information.High quality and high resolution image can be obtained by using the low resolution video sequence inversion through video super resolution reconstruction technology.The paper's main work is as follows:First,we study the joint acquisition,correlation and access of video and navigation information,and design a 4 channel video real-time transmission system.Based on the embedded development board firefly,Using 264 cameras to capture video,using ten axis sensor package to capture navigation information,then data matching,packaging,RTP transmission.At the receiving,the client software is designed to decode the received data and analyze the sensor data,which are used in vehicle navigation and measurement.Second,for the multi camera video capture,a video mosaic algorithm based on ORB feature points is proposed.For the video frames per second,the feature points are detected and matched only in the first frame,RANSAC method is used to extract the feature points and estimate the transformation matrix.The following frames use the optimal homography matrix of the first frame estimation to directly splice.Third,through the research of video super-resolution technology,aiming at the video ROI(Region of Interesting),Based on the analysis of the deficiency of the traditional total variation regularization model,an improved weighted two parameter regularization model is proposed.A priori information constraint on the total variational regularization term is obtained by using differential curvature and structure tensor;in the super resolution iterative solution,different regularization parameters are used to constrain the smooth and non smooth regions;It not only retains the details of the edge and texture,but also has a strong ability to denoise.