Video Analysis Technology Based On SPARK

With the progress of cloud platform and the development of related technology,more and more tasks are delivered to computer cluster.And the processing of huge amounts of video data expresses the representative character.Parallel frameworks such as Hadoop and Storm offer Parallel Execution plan for that.As one of the latest distributed framework,Apache Spark has a perfect compatiblity with Hadoop,and also a fine-augmentability.With the help of RDD,many programs will be operated in the memory,which will make a substantial increase in efficiency.As a subsystem of Spark,Spark Streaming can turn streaming data into bath data.In that way I put forward a video analysis system based on Spark in this paper.Among a wide variety of video analysis methods,a key frame extraction method aimed at surveillance video data is projected in this paper.It uses the frame averaging method and the Gaussian Mixture Model in order to initialize the back ground model,and then gets the key frame by target motion analysis.The problem of how to make the algorithm into MapReduce method and keep the consistency of the targets is also settled in this paper.And then this paper recommends how to input data into DStream and the relationship between Hadoop HDFS and Spark Streaming.In the end,based on the data of the experiment,a performance optimization scheme,which is in the way of modification the configuration parameters of Spark Streaming,is also proposed.And the experiment is regard to compare the run time of the algorithm working on local system and Spark system,which also test and verify that the parallel algorithm is viable and efficient.