Research And Application Of Cloud Transcoding Of Streaming Media Based On Spark

With the popularity of smart phones and the coverage of 3G/4G network,mobile Internet technology is gradually penetrated into the lives of the people.And the decline in mobile network tariffs,so that users are gradually starting to watch the video and network broadcast live through mobile devices.But because the mobile network signal instability,will cause the user to watch high-definition video footage which is jammed frequently,it can only reduce the clarity to get fluency,so the traditional streaming media technology can not meet fluency of user and clarity of both requirements at the same time.HLS(HTTP Live Streaming),a new generation of streaming media transmission protocol,is issued for the Apple Corp in 2009,HLS is not only to make up for the traditional streaming media protocol shortcomings,but also provides the adaptive multi rate,that can automatically switch in different network environment to guarantee the smooth playback resolution.The implementation of the HLS protocol requires that the video conform to its MPEG-2 TS encoding format.Since the transcoding is very resource-intensive Task,and now the mainstream single-core transcoding has been unable to meet the requirements of the people.At the same time,the rapid growth of video demand,making the mainstream of the centralized single transcoding seems stretched,the cost of providing video services is also high,so distributed and parallel processing is the inevitable trend of video transcoding.This paper uses Spark open source distributed computing platform,through Spark's memory calculation can be more than Hadoop-based cloud transcoding system faster.In addition,Spark provides Spark Streaming streaming computing capabilities,can be convenient and efficient handling of streaming video,video broadcast capabilities.In this paper,we design the whole framework of streaming media cloud transcoding system,which mainly consists of transcoding module,Spark module and streaming media server module.In order to achieve transcoding function,transcoding module use open-source video processing software FFmpeg.Spark module has achieved two functions: First,through the framework of Map Reduce,transcode module carried out transcoding to achieve video on demand function;second,through Spark Streaming streaming,completing the video stream transcoding and live functions.Streaming server module using open-source HTTP server Nginx to build a simple HLS server in the content distribution part,and by it to complete the video clip,add index and other resources consumption of small post-work.At last,this paper tested and analysed the streaming media cloud transcoding system which has been achieved.Compared the performance of Spark,Hadoop and Spark in different conditions of Streaming,and then test the effect of the sub node number of transcoding performance,experiments were carried out under the ideal conditions of the system when the live video broadcast delay.