Research On Fast Video Retargeting Based On Seam Carving

With the growth of videos from various devices with different resolutions and aspect ratios,how to adapt video to a variety of mobile devices has become an important research topic.Video retargeting is to resize a video to a desired resolution or aspect ratio.traditional video scaling methods,such as uniform scaling or cropping,usually distorts or discards salient objects.Content-aware video retargeting,which resizes videos or changes their aspect ratios while preserving the shape of salient objects,provides satisfactory watching experience for the same video content on different viewing devices.Content-aware video retargeting methods are generally divided into two categories: seam carving methods and grid-based methods.Seam carving methods take into account the consistency of the current frame and adjacent frames,finding the best seam to retarget the video,while grid-based methods independently divide each frame into grids and align the grids between consecutive frames,each grid is aligned to the corresponding ones in other frames.In recent years,researchers have proposed a number of video retargeting methods and achieved good results.However,there are still many shortcomings in the current video retargeting methods:(1)seam carving methods usually have better effect on single frame processing,and can better maintain the spatial coherence of the single frame,however,they can't preserve temporal coherence well between frames,resulting in video artifacts and jitter.In addition,the seam carving methods are too timeconsuming due to high computational complexity of the dynamic programming.Most seam carving methods can't coordinate well with spatio-temporal coherence,and are less effective when dealing with videos that contain high-speed moving objects.(2)Grid-based methods typically maintain good temporal coherence across multiple frames,resulting in less video jitter.However,due to the coarser processing granularity(based on the grid),these methods produce more distortion in the video.In addition,although the grid-based method is faster in processing,it usually needs to process all video frames at one time,so it cannot be applied to real-time video streams,which leads to great limitations in application.Grid-based methods have better results in processing video with low-speed moving objects,these methods can't preserve temporal coherence well for videos with high-speed moving targets.Aiming at the problems existing in current video scaling algorithms,this paper intends to carry out research on video retargeting technology based on seam carving.The main research contents of this paper include:(1)Energy function based on spatio-temporal coherence for video retargetingAt present,most video retargeting methods based on seam carving use continuous seams to change the video resolution.However,the drawback of this method is obvious.For two separate frames of video,although consecutive seams between frames can maintain temporal coherence,it is also possible to cause distortion of salient objects in the next frame,resulting in video artifacts or jitter.In order to solve these problems,we propose a new energy function based on spatio-temporal coherence,which avoids the generation of video artifacts through the seam found by the function.(2)Video retargeting based on genetic algorithmOne of the core steps of a seam carving video retargeting algorithm is to find seams to delete or add.Dynamic programming algorithm is usually used to find the seam with the lowest energy value on the energy map in current methods.However,dynamic programming requires scanning all possible paths on the energy map,and the time complexity is very high,resulting in inefficient algorithm execution.We observe that for video retargeting,the sub-optimal seam is similar to the optimal seam.Therefore,we use genetic algorithm instead of dynamic programming algorithm to perform seam search process,which greatly reduces the time complexity of algorithm operation while ensuring the execution effect of the algorithm.Compared to the state of art method,our algorithm performs three times faster.In summary,this paper aims to develop a video retargeting algorithm based on seam carving.In view of the fact that the current methods based on seam carving can not coordinate the spatio-temporal coherence,this paper proposes a new spatiotemporal coherence energy function.In addition,in order to solve the problem of time complexity of the seam carving video retargeting method,this paper uses genetic algorithm instead of dynamic programming to perform seam searching process,which greatly improves the running speed of the algorithm.For different types of video,this paper sets different genetic algorithm parameters,so that the algorithm can be applied to many different types of video.