Design And Implementation Of Data Acquisition And Processing Systems Based On Edge Computing

With the rapid development of electronics,information and communication technologies such as the Internet of Things,5G,blockchain,Connected Vehicle Network,and sensors,the growth of various types of data volume has become an exponential trend,and the requirements for massive data processing capacity and processing speeds have also increased.Cloud computing technology provides users with almost unlimited computing power through a large number of high-performance servers in the data center,and is one of the important solutions for big data analysis and processing.However,cloud computing also has a series of problems such as large amounts of network delays,high costs,and security,and it cannot adapt to all big data analysis and processing requirements.For example,in the scenes of industrial production and operations,real-time response to accidents,failures,and unexpected events is very important.In the network data capture scenario,data transmission costs are more sensitive.In order to solve the above problems,this paper researches and analyzes cloud computing and big data technology,proposes a data acquisition and processing system architecture based on edge computing,and implements system design and implementation for two specific scenarios.Utilize the edge computing/storage equipment close to the user and combine with cloud computing to provide a low-latency,high-processing capacity data acquisition,processing,and analysis solution.The main research results of this thesis include the following aspects:First of all,this paper analyzes the existing technology and research status,compares the existing problems in the actual use of cloud computing technology,and discusses and designs a new software and hardware scheme of edge computing framework combined with edge computing and cloud computing center and the related way of communication.Finally,the overall scheme for edge computing using the Raspberry Pi as an edge device and connecting via Socket communication was established.The system can meet the task requirements of a variety of application scenarios,enhance the extensive use of equipment and save costs.Secondly,according to the scene of human behavior recognition,a human-behavior data acquisition,processing and analysis system based on edge computing is designed and implemented.In this scheme,after the human behavior data collected by the IMU is cleaned and structured,the behaviors of the corresponding monitoring objects are classified and modeled using Xgboost,and then correcting these models using distributed computing from edge devices,thus,a human behavior recognition system with high efficiency,low energy consumption,and quick response is realized,and the applicability of the current computing framework to physical data processing is verified.Finally,according to the massive data in the Internet,a literature data crawling and processing system based on the edge computing framework is designed and implemented.By slicing too large a crawling task that cannot be completed in a short time by a single device into multiple small tasks and distributing it to the edge devices under the control of the cloud computing center,the speed of network data collection is accelerated.In addition,through extracting relevant information of all documents stored on the IEEE Xplore website and storing it in the cloud,the system demonstrate the ability of capturing and processing large amounts of network data in the current network environment,also verified the feasibility of the proposed edge computing architecture.