Distributed Invasive Tumor Growth Optimization Algorithm Based On Cloud Computing And Its Application

With the advent of the era of Big Data,the complexity and scale of many optimization problems are constantly increasing.It leads to a huge search space which makes search easily get in a local optimum and imposes a heavy computation cost causing lots of time consuming,which poses severe challenges to traditional evolutionary algorithms.Invasive tumor growth optimization(ITGO),a new evolutionary algorithm proposed by this laboratory,has been proven to have excellent search ability and practical application value,but due to the lack of scalability,it is difficult to find the optimal solution in a reasonable time when addressing some large-scale optimization problems.Cloud-oriented distributed computing is widely used because of its advantages of flexibility,easy scalability and convenient processing for large-scale data.Evolutionary algorithms can be migrated and expanded to cloud computing platforms through distributed computing frameworks to achieve parallel optimization requirements.Thus,combing the ITGO algorithm and the cloud-oriented Spark framework,this thesis designed and implemented a distributed invasive tumor growth optimization algorithm,Spark-ITGO with good scalability and search ability,so that it can be applied to solve large-scale optimization problems.Combing the features of island model and Spark platform,a general Spark island framework based on Resilient Distributed Dataset(RDD)and central broadcast mechanism was designed to parallelize the ITGO algorithm,enabling ITGO algorithm to run on the cluster of multiple-node and multiple-core in parallel,thereby accelerating the evolution of the entire population.With universality and scalability,the framework does not require any modification to the original evolutionary algorithm and can be extended to parallelize most evolutionary algorithms.Combining the features of the serial ITGO algorithm,a balanced multi-island optimal migration strategy was designed to migrate individuals among multiple subpopulations to achieve information exchanges.The strategy considers both the introduction for diversity of population and the quality of migrated individuals,which also can maintain the overall quality balance of the immigrants among multiple islands.It can effectively increase the diversity of population and avoid converging into a local optimum,thereby promoting the global search ability and accelerating entire the convergence process.In this thesis,Spark-ITGO was applied to solve specific application problems such as clustering of biomedical data,container scheduling in cloud datacenter and epistasis detection of associated genes.According to the specific application scenarios,Spark-ITGO was improved,including discretization of solution space,adding real constraints,etc.to further improve the effect to address real-world problems,thereby verifying the ability of Spark-ITGO to solve practical optimization problems.Spark-ITGO was evaluated by many benchmark tests,scalability tests and real-world application experiments on a multi-node cluster,and was compared with ITGO and other optimization algorithms.The experimental results show that with good scalability,Spark-ITGO's ability to search optimal solutions is further improved than ITGO and can be effectively applied to address real-world problems.