Research On Video Retargeting Based On Frame Alignment And Grid Deformation

With the continuous development of multimedia technology,a variety of video terminal equipment such as mobile phone,mobile pentium 4,computer,high-definition television and so on emerge in an endless stream.Different devices also have different screen sizes and ratios of width to height,which presents a new challenge--how to change the resolution of the original video to meet the display needs of different devices.The traditional three methods of cropping,letterboxing and uniform scaling ignore the difference of video content,which leads to distortion of the significant part of video content,therefore,content-based video retargeting method is proposed.The main purpose of content-based video retargeting algorithm is to protect the significant part of the image from deformation and avoid jitter between video frames during video retargeting,and the algorithm should be efficient.The results of existing content-based video retargeting algorithms are still not ideal: most of the retargeting methods that solve video frames sequentially only consider the content correlation between adjacent two frames or adjacent frames.This method has low computational complexity,but it is difficult to maintain spatial and temporal continuity of the resulting video.While the multi-frame video redirection method can get better redirection results,it is difficult to process video stream in real time due to its high computational complexity.In addition,when the video background contains some features that have an impact on the watching video of human eyes,the retargeting algorithm based on grid deformation will cause discontinuity of the spatial content.In order to solve the above problems,this dissertation makes an in-depth discussion and research on the video retargeting algorithm based on frame alignment and grid deformation.The research results are as follows:An video retargeting algorithm based on frame alignment and grid deformation is proposed.On the premise of obtaining the significance region and the motion significance region of the image,the algorithm takes the first frame of the video as the benchmark,and aligns the video frames in camera coordinates by estimating the camera motion between frames,so as to obtain the global motion history map.Then,the motion history map is used to guide the single-frame image deformation sequenced,and the problem of the single-frame image shrinkage is transformed into the grid deformation problem.The optimization model of maintaining the overall shape on the grid is established by constructing a series of constraints,and the deformation grid is finally solved to obtain the retargeting video.The experimental results show that the method in this dissertation can obtain better retargeting results for both stationary video and video with complex motion,and can effectively maintain the shape of moving objects while maintaining good time continuity of video background,and can realize real-time processing of video stream.Since the video retargeting algorithm based on grid deformation compresses the deformation part into the background as much as possible,if the background contains some information that has an impact on the watching video of human eyes,the spatial content will be discontinuous.To solve this problem,a multi-operator video retargeting algorithm based on grid deformation and intelligent cropping is proposed in this paper,which can effectively reduce the spatial content distortion caused by grid extrusion,so as to output better video with spatial content continuity.Specifically,first of all,the video frames are aligned under the camera motion model to obtain the motion history map,according to the motion history map here,each video to determine the different cropping window,and then according to the important degree of each video frame diagram,draw lessons from the dynamic programming algorithm to find a optimal cropping path.After the cropping window and the cropping path are obtained,the video frame and the importance map are cropped,the corresponding scaling factor from the cropping sequence to the retargeting video size is converted,and the retargeting operation is carried out on the basis of grid deformation.The experimental results show that the video retargeting algorithm proposed in this dissertation can effectively reduce object distortion caused by grid deformation,and the obtained results can better protect the important areas of the original video from deformation under the premise of maintaining good time continuity of the retargeting video.