Multimodal Video Scene Segmentation Algorithm Based On Genetic Algorithm

With the rapid development of computer,multimedia,network and other technologies,the multimedia information(especially the vi deo data)grows rapidly as one of main sources of information access.Facing with vast video data,the urgent problems include: how to manage them effectively,how to locate interesting video information rapidly and accurately and how to understand main co ntent of some video quickly.In this case,the video scene segmentation is an important link of the content-based video retrieval as a research hot spot.On the basis of the sufficient analysis of video content and in-depth study on video scene segmentation technology,this thesis puts forwards a multimodal video scene segmentation algorithm based on the genetic algorithm.Firstly,three multimodal low-level features are extracted in video shots,which includes the image feature,the audio feature and the t ext feature.And there is temporal associated co-occurrence(TAC)characteristics among three multimodal low-level features,which influence and complement each other.Therefore,the choice and fusion of multimodal features can reflect the content of video shots accurately and improve the accuracy of the scene segmentation.Secondly,the similarity is measured among shots.In order to make full use of TAC characteristics and improve the accuracy of calculation results,this thesis calculates the shot similarity using the indirect calculating,in which the similarity of the same modal data is calculated firstly,and the correlation of different modal data is calculated then,and finally the ultimate similarity is gained by fusing the similarity and the correlation additively and kept in the shot similarity matrix.Finally,a multimodal video scene segmentation algorithm is put forward based on the genetic algorithm,combining the thought of multimodal feature fusion and improving from the initial population generation method,the evolution progress and the termination condition of genetic algorithm,which aims at a lower precision and a slower convergence rate in the scene segmentation algorithm based on genetic algorithm from Xiao Z.M.The algorithm calls the collection of L continuous shot in video as an individual,taking the segmentation point in each shot as a gene,in which the individual is encoded with the binary encoding and the encoded individual is a specific manifestation of scene classification.Thi s thesis gains an initial individual with the shot similarity matrix,and then takes a 16-individual population,bred by crossover and mutation from this initial individual,as an initial population.Starting from this initial population,this thesis evalu ates individuals through a reasonable function,and completes the evolution course using genetic operators,which makes the genetic algorithm evolutes in the direction of the optimal solution.Till the termination generation is reached and the evolution st ops,it will output the individual code with the highest fitness in contemporary populations,and according to this code,the decoding is conducted for a scene boundary.Next,it conducts a merging treatment for segmented scenes aiming at the over-segmentation phenomenon.The results show that this proposed algorithm can segment the video scene better with different effects for different types of videos.As a whole,the comprehensive measure index F reaches 87.3%,higher than the scene segmentation algorithm based on entropy and SURF(Speeded Up Robust Features)from Baber J.,et al by 7.4% and the algorithm from Xiao Z.M.by 5.1%,and the recall and precision reaches 86.9% and 87.7% respectively.