Font Size: a A A

Research On Highly Efficient Data Processing Framework Based On Container

Posted on:2019-11-29Degree:MasterType:Thesis
Country:ChinaCandidate:J YinFull Text:PDF
GTID:2428330590492302Subject:Computer technology
Abstract/Summary:PDF Full Text Request
Mobile Internet has entered a period of rapid development with an exponential growth in the number of users.The distribution of users is also rapidly expanding,and user requests have also grown rapidly.However,the bandwidth of network transmission is growing slowly.In this context,the traditional data processing model has reached a "bottleneck." Although cloud computing solves the bottleneck of computing power under big data,it is mainly applied to offline batch processing applications and is greatly affected by the network transmission bandwidth.In today's network bandwidth capabilities,real-time applications for a wide range of small amounts of data do not result in significant response delays and significant improvements in hardware resource utilization.Edge computing is an emerging trend in recent years,it relies mainly on edge devices close to data sources.In this computing model,each edge device has some storage and computing power,so it can not only pre-process or analyze some data in the cloud,but also handle real-time applications with small data volume.Compared with traditional virtual machine technology,container technology has significantly improved energy consumption and hardware resource utilization.The most popular in real-time applications today is streaming computing,such as Storm,Spark Streaming,and more.According to the distribution characteristics of edge devices and their own resources,the distributed task flow based on streaming ideology can improve the flexibility and efficiency of edge networks.In this paper,we study a dual-domain scheduling strategy that aims to combine edge computing,container technology,and distributed task flow,and according to the distribution and heterogeneity of edge devices and the distribution of task flows,Multi-level load balancing high-performance data processing network.With the iFogSim simulator,the experimental results show that the related technology can improve the ability of real-time user requests processing.
Keywords/Search Tags:Cloud Computing, Edge Computing, Container Technology, Streaming Data Processing, Resource Scheduling
PDF Full Text Request
Related items