Design And Implementation Of A Parallel Computation Framework Based On Lightweight Virtual Machine

With the development of sensor technology and communication network,the scale of data collection and storage is increasing rapidly.How to mine the effective information from the massive data is the current research hotspot.At present,the most common solution is to use cluster system and distributed framework to improve the efficiency of data processing.Parallel computing is an effective way to improve the computing speed and processing power of computer system.MPI is widely used in the field of academic research,but in the commercial field,the cloud computing system mostly adopts MapReduce parallel programming model.MPI is the main programming model for developing parallel applications--the fact standard of message passing programming model.However the MPI code reuse rate is low,expanding machine performance is poor,demand for programmers is high.MapReduce is a parallel programming framework proposed by Google.However,it is difficult for users to deal with complex data,users have to write a large number of MapReduce tasks,the results tend to result in low efficiency.Therefore,in order to solve the above problems,this paper proposes a parallel computing framework based on lightweight virtual machine,The framework uses a lightweight virtual machine Contiki OS as the computing unit.Based on the theory of parallel computing framework,this paper analyzes and studies Hadoop's distributed computing model MapReduce and distributed file system HDFS,analyzes the advantages and application value of lightweight virtual machine Contiki system,and designs based on lightweight Virtual machine Contiki system parallel computing framework,the framework without MapReduce,consumes less resources,completes the framework of the lightweight virtual machine design,client modules,service node module,control node module,computing node module and parallel strategy part of the key technology design,designs load balancing algorithm based on the load weight,and in the Linux environment to achieve parallel computing framework.Finally,the test environment is set up,and the performance of the different computing nodes and the influence of the number of different computing nodes on the performance are analyzed by experiment and serial calculation.The performance and resource consumption under the same calculation task are analyzed by comparing with Hadoop.It can be seen that the frame is faster than the serial calculation,use less resources compared to Hadoop.