| With the development of the Internet,users can freely publish video content on the Internet platform.These content may have a variety of violations,and the platform needs to audit it.Under traditional conditions,manual audit of video content is the main method.This method has high accuracy.However,with the development of platforms and the increase in the number of user-generated content,the high cost and low efficiency of manual audit can no longer meet the requirements.Therefore,the enterprise platform needs an automated video content audit system that can maintain accuracy and high performance in scenarios with large data volumes and high concurrency.The current emerging trend is to use neural networks to identify violations in video content.This is the core work of automated audit.However,existing automated audit systems have the following three problems in enterprise-level applications.First,the accuracy of identifying violations of video content is not high.Existing audit systems use a single neural network to audit the overall content of the video,but the video contains multi-dimensional content such as text,audio,and images,and the types of violations contained in it are complex.It is difficult to cover all violation issues of multi-dimensional content in the video by a single neural network.Second,the system performance cannot support high concurrent requests of the magnitude of big data.Existing audit system has the defects of long audit delay and high resource consumption in the face of 100,000-level daily throughput.Third,the existing audit system lacks access control.The system cannot identify and manage business parties,and business parties can access the review system at will.Unapproved access requests will waste system resources and seriously affect the normal audit process.Based on the above background,this thesis designs and implements a video content audit system for high concurrent requests.This system uses the microservice model and the responsibility-chain model.In order to solve the above problems,this thesis constructs four services,including video content audit service,video stream preprocessing service,access control service,and memory cache service.The details are as follows:·Aiming at the problem of low accuracy in identifying violations,this thesis designs and implements a video stream preprocessing service and a video content audit service.The video stream preprocessing service splits the video content,including extracting image frames from the video,extracting audio and converting it into text.The video content audit service distributes the above-mentioned split content to different neural networks,and each neural network only identifies specific types of violations in specific dimensions of content,and after obtaining the violation identification results of the split content,it combines the existing video overall content violation identification.Integrating all the recognition results can effectively improve the accuracy.In addition,the video content audit service collects audit raw data and result data as training data for subsequent training of neural networks to continuously improve the accuracy of violation recognition.·Aiming at the problem of insufficient system performance under high concurrency,this thesis designs and implements a high-performance video content audit service and a memory cache service.Video content audit services use message queues for asynchronous communication to reduce congestion;use Redis database to cache data packets to reduce access time;use responsibility-chain model to abstract long links into task chains,and multithreads to run concurrently and asynchronously to improve system audit performance.The memory cache service caches system metadata in the memory to avoid performance bottlenecks caused by frequent system requests to the database.The service uses triggers and timers to ensure the timeliness of the memory cache.·Aiming at the problem of lack of access control in the system,this thesis designs and implements an access control service,which is used to manage the business parties which need to access,filter unauthorized access requests during audit,and ensure the normal process of audit services.In addition,this service can also provide the business side with customized configuration capabilities,and provide customized feedback based on the business side’s configuration in the video content audit service.The video content audit system has been put into operation on well-known domestic Internet platforms.It is currently operating stably,providing services for 200 business parties,with a daily throughput of up to 500,000. |
