Design And Implementation Of Similar Video Retrieval System Base On Video Fingerprint

With more and more video application scenarios on the Internet,such as variety shows,short videos,news,etc.,the number of video resources is increasing rapidly.A large amount of video resources flood into the database,which is filled with a lot of duplicate data.Some users repeatedly upload other people's original works in order to earn clicks and related rankings.This kind of behavior seriously damages the interests of the original author,and also affects the user's experience when browsing information.Many research institutes and companies have invested a lot of energy in the field of video deduplication,such as hashing techniques for video processing and binary watermarking.However,these solutions are based on the binary nature of the data and are not related to the video content,so the results are not satisfactory.We propose a similar video retrieval system based on video fingerprinting.The video fingerprinting technology is used in the system.The so-called fingerprint technology refers to processing video to obtain feature data related to the content.A similar video can be confirmed by matching the new feature data with the old feature data.Based on the requirements of the syst em,this paper takes the macroscopic design and implementation of similar video retrieval system as the starting point,analyzes the main functional requirements and performance requirements of the system,points out the main performance bottlenecks and technical difficulties of the system,and addresses each problem.Perform analysis and architecture optimization.From the perspective of information technology,the system design,service compatibility standards,program processing flow of each subsystem,data format between modules,etc.are introduced to provide method guidance and theoretical reference for the following specific implementation.The similar video retrieval system consists of four parts: the external agent system,the feature extraction system,the feature storage system,and the feature index establishment system.The external agent system is the only interface that provides services externally,and is responsible for receiving and responding to requests and invoking the query process.The feature extraction system is responsible for fingerprint feature extraction of the video,and the corresponding content vector is processed by using the correlation algorithm to process the video content.The feature storage system consists of a real-time library and a full-scale library.The real-time library stores only the new video features generated in the near future.The full-scale inventory stores all the historical video feature data in the form of an index file.The module provides feature matching retrieval.The feature indexing system acts between the real-time library and the full-scale library,and converts the lightweight multi-dimensional feature data into a heavy-weight index file to improve the search efficiency of the full-scale library.After functional and performance testing,a similar video retrieval system was able to retrieve highly repetitive videos in the database,enabling efficient deduplication of similar videos.The architecture and implementation of the system meet the preset functional requirements and are compatible with other services.In terms of performance,the system handles small video in less than 40 s,single frame image processing time is less than 400 ms,and the overall system concurrency is 6 requests per second.The above effects meet the business needs,and the system is running stably after being online.