Research On Video Copy Detection Algorithm Based On Spatio-temporal CNN Features

The existing video copy detection algorithms can not effectively combine the spatial and temporal features of video,so there is still a lot of room for improvement in detection performance.Although some of the latest algorithms based on convolutional neural network(CNN)features outperform the traditional ones based on manual features in detection performance,their detection performance is still low due to the failure to deal with the relationship between video spatial domain features and time domain features.In view of the above shortcomings,this paper proposes two video copy detection schemes based on CNN features in spatio-temporal domain.1)A video copy detection algorithm based on CNN features in spatio-temporal domainIn order to solve the problems of low detection accuracy and low detection efficiency in existing video copy detection methods,this thesis proposes a novel video copy detection algorithm.Since the amount of video data is usually very large,in order to reduce the complexity of data calculation,this algorithm samples the video at equal intervals firstly.Then use CNN network to extract compact spatial domain features for the sampled video frames.In order to describe the video more accurately,this algorithm calculates the centroid of the convolutional feature map output by CNN and encodes the tempora domain feature.As a supplement to the characteristics of the spatial domain,this feature can make up for the lack of the ability to describe the spatial domain features.In the copy detection stage,the spatial domain features is firstly used to filter out the suspicious copy video,and then the temporal domain features is used to filter the wrong matching results to obtain the final copy detection results.Since this method uses time domain features for post-validation to remove false matching results,the detection accuracy is high.Moreover,the use of CNN features makes the extracted features compact enough,thereby improving the speed of copy detection while improving the accuracy of detection.2)A video copy detection algorithm based on spatio-temporal visual words and hierarchical indexIn order to further improve the performance of existing video copy detection algorithms,this thesis proposes a video copy detection algorithm based on spatio-temporal domain visual words and layered inverted index.This algorithm extracts robust SURF descriptors(Speed-Up Robust Features)from the sampled video frames as the spatial domain features,and quantifies the SURF features through the bag-of-words model to obtain the spatial domain visual words.For the problem of unstable centroid generated in the previousalgorithm,this algorithm uses the stability of SURF points and encodes the SURF point displacement information between video frames to obtain multiple layers of temporal domain visual words.In order to achieve efficient detection,the visual words in the spatio-temporal domain are stored in the inverted index structure based on the multilayer visual words proposed by this algorithm.In the copy detection stage,in order to distinguish the different description capabilities of spatio-temporal visual words,the matching similarity between the layers of spatio-temporal visual words is obtained by layered calculation.The final similarity between the two videos is calculated by similarity fusion calculation to determined whether the current video is a copied version of the original video.A large number of experimental results verify the excellent performance of the algorithm in detection accuracy and detection efficiency.