Research On Key Problems Of Efficient Processing Of Big Data In Cloud Computing

Big data has become a research hotspot in the field of information technology and other interrelated interdisciplinary fields.It is considered as an important and valuable asset and has great potential to promote people's production and development of science and technology.By analyzing,processing,developing and utilizing big data,new knowledge,new rules,answers to questions,and even predicting models can be obtained that have important positive impacts on research,production and life.However,the characteristics of big data(such as large-scale,real-time,etc.)make the current widely used centralized computing architectures and classic data analysis models and algorithms cannot be directly applied to big data to achieve effective and efficient analysis and processing for big data.This is mainly due to the complexity of big data itself,and the current mainstream computing paradigms are difficult to provide flexible,scalable and adequate computing capacity for efficient processing of big data.The emergence and development of cloud computing in recent years has provided the possibility to solve this problem.Cloud computing has gradually become the basic underlying platform for research and application of big data.This thesis divides the research on the efficient processing of big data under cloud computing into the following three directions: cloud computing platforms,big data preprocessing and classical data processing algorithms and models.First,in order to efficiently handle big data,we need to use the virtualized parallel computing capabilities provided by cloud computing platforms with elastic scalability to handle big data computing tasks.Second,we can perform targeted preprocessing on big data itself before analysis to big data to achieve a multiplier effect.Therefore,it is necessary to investigate efficient preprocessing approaches of big data under cloud computing.Third,the existing classical data analysis algorithms and models should be combined with the cloud computing technology for redesign research on big data problems,and thus to achieve efficient processing of big data,rather than using traditional algorithms and models directly for big data.This thesis refers to the research problems involved in the above three directions as key common problems in the field of efficient processing of big data under cloud computing.This thesis has mainly focused on and investigated a number of key problems of efficient processing of big data in cloud computing.This thesis has studied multi-task deployment approaches and live virtual machine migration policies for efficient processing of big data in cloud computing;studies mobile cloud computing models and configuration approaches for efficient processing of big data;studies big data instance reduction preprocessing approaches and big data recommendation system frameworks for efficient processing of big data in cloud computing.The main contributions of this thesis are as follows:(1)This thesis has conducted research on multi-task allocation approaches and live virtual machine migration policies for efficient processing of big data in cloud computing.In order to take full advantage of the powerful parallel computing power supported by virtualization and elastic scalability of resource pools of cloud computing to obtain underlying infrastructures for efficient processing of big data,efficient multi-task provisioning approaches and live virtual machine migration policies in cloud computing should be studied and implemented first.Thus,this thesis has proposed a novel heuristic multi-task provisioning approach called LB-BC(Load Balancing approach based on Bayes theorem and Clustering),which is based on clustering analysis and Bayes theorem for long-term load balancing optimization,and an energy-aware heuristic placement selection approach of live virtual machine migration,which is called PS-ABC(Placement Selection policy based on improved Artificial Bee Colony and bayes)and is based on an improved artificial bee colony algorithm and Bayesian probability theory.The LB-BC approach has realized the long-term load balancing optimization of cloud computing platform for big data with less overhead,improved the external service capability of the cloud platform,and promoted efficiently parallel processing of big data under the cloud computing.The PS-ABC approach has realized the efficient and energy-saving live virtual machine migration policy under the cloud platform for big data.With the guarantee of the success rate of live virtual machine migration events,PS-ABC has combined with live load balancing migration optimization of virtual machines to achieve overall performance improvement of the cloud platform for big data.To some extent,they have laid the groundwork for the efficient processing of big data in cloud computing.(2)This thesis has conducted research on mobile cloud computing models and configuration approaches for efficient processing of big data in cloud computing.In order to explore the efficient processing of big data under the cloud computing from the perspective of exploring big-data edge computing,this thesis has studied mobile cloud computing models and the corresponding configuration methods for efficient processing of big data.First,this thesis has presented a novel mobile cloud computing model which is designed for parallel computing on the mobile end and has combined remote cloud storage with mobile virtual cluster.And the corresponding heuristic configuration approach,called VD-ABC(Virtual device Deployment based on Artificial Bee Colony),for deploying multiple virtual machines on the mobile end has been put forward.This mobile cloud model is intended to make full use of computing resource of multiple devices on the edge mobile end.It can maximize the corresponding resource utilization and achieve efficient parallel computing by virtualized multi-tenancy.By using the proposed VD-ABC approach,one can obtain the virtual machine deployment solution that can save energy and optimize service response delay of mobile devices with guaranteed performance.And the achievement of efficient processing of big data based on virtualized edge cloud computing will be facilitated.Second,this thesis has proposed a mobile cloud computing model based on multi-device collaborative computing that combines dynamic mobile ad-hoc community and mobile application partitioning.It uses a large number of mobile idle devices to form a logical mobile community as an edge cloud computing cluster.By deploying and executing multiple mobile application partitions in multiple devices,it can actualize parallel processing abilities for big-data processing requests.Correspondingly,this thesis has proposed a heuristic mobile application partitioning dispatch approach called MCC-PSO(Mobile Cloud Computing based on Particle Swarm Optimization),which is based on improved particle swarm optimization,under the proposed mobile cloud computing model.The MCC-PSO approach has shortened the overall processing time of multiple mobile application partitions and reduced the consumption of remaining electricity of mobile devices under the premise of satisfying the multi-task parallel computing performance.Thus,the mobile cloud model as a whole has the potential to facilitate efficient processing of big data based on self-organizing edge mobile cloud computing.(3)This thesis has studied big data instance reduction preprocessing and big data recommendation system frameworks in cloud computing.Aiming to study efficient processing of big data under cloud computing from the perspective of big data preprocessing in cloud computing,this thesis has proposed a heuristic instance reduction approach called CSA(Clustering Sampling Algorithm),which is based on clustering analysis and the idea of optimal minimum sample set in cloud computing.The CSA approach can effectively and efficiently achieve instance reduction preprocessing of big data through sampling of optimal minimum sample sets of all clusters while ensuring distribution characteristics and information quality of original big data.To a certain extent,it has directly and substantially promoted the realization of efficient processing of big data in cloud computing.In addition,this thesis has proposed a novel big data recommendation system framework approach called BDRSF(Big Data Recommendation System Framework),which combines big data and social context theory under cloud computing.The BDRSF approach can achieve the corresponding recommendation prediction performance.From the perspective of common frameworks of efficient prediction models for big data under cloud computing,it has promoted the realization of efficient processing of big data.