The Improvement And Research Of Three-dimensional DEM Simulation Software Based On MPI

There are many particle objects that have different shapes and sizes in the widereal world’s three-dimensional space, such as the soil and sand particles on the ground,suspended particles that will do damage to human’s health in the air like pm2.5,organic carbon POC that are widely spread in the sea, and all kinds of crops likesoybean, millet and corn etc. The researches on the movement properties of particlesubstances are helpful for us to recognize the rules of objective material’s movement,change the world, improve our labor productivity and standard of living.The property of particle objects is between liquid and solid, from the1970s,manyphysicists and mathematicians throughout the world have conducted studies anddiscussions about the movement principles of particle objects. Among them there is afamous professor named P. A. Cundall, his discrete element method has become ageneral method that is used to analyze particle substances’ mechanics movement.The core idea of discrete element method is: the simulation object system isregarded as a combination of some small particles, every particle is an independentunit that has quality, shape, speed, position and other physical attributes, according tothe relationship of an object’s acceleration, quality and force, which is revealed by theNewton’s Second Law formula F=ma, we can calculate all the particles’ differentcontact and movement situation, thus analyze the movement process of the wholesystem in a specific period.On the basis of the study of discrete element method for many years, ourresearch team have developed a three-dimensional DEM software called AgriDEM,which is updated and maintained continuously and mainly applied in the area ofmechanical engineering. Now this software can simulate the interaction process ofdifferent mechanical components and many spherical particles or non-sphericalparticles. The result file data that can be gotten after the calculation, play an importantrole of reference and assist in the design and improvement of machinery products. To improve the performance and efficiency when this software is executingdiscrete element method simulation, and expand the scale of calculation, our researchteam has tried in many ways. Formerly we have applied OpenMP multi-core paralleltechnology based on shared memory successfully, execute the calculation task parallelin some parts of the program that can be multi-thread accelerated, in order to fullyexploit the multi-core processor’s potentialities and shorten the simulation time.But according to the period trends of the calculation capability’s ascension ofcomputer hardware that is revealed by Moore’s Law, the number of the cores of aprocessor can’t increase unlimitedly, they will compete for the memory space moreseriously even if after the increment; moreover, not all the procedures are suitable formulti-core parallel, such as in order to keep the consistency of data in the memory, thewrite and read operation on shared data need to lock and unlock the variablefrequently. So we need to find other methods to improve the calculation speed andefficiency of the AgriDEM program.Therefore we focus on the distributed parallel technology, hope to organize aparallel computing cluster consists of some computers with multi-core processors,which is connected by a local area network. then execute the three-dimensional DEMsimulation program that supports distributed standard to realize multi-computerdistributed parallel computing. This paper’s main work is about solving the problemsthat appeared in the process of rewriting previous program with distributed parallel,including how to divide the calculation tasks among the nodes in the cluster, how tostore the messages related to simulation calculation separately, message passing andcommunication dispatching method, how to revise the former algorithms and datastructures and so on.This paper firstly introduces AgriDEM software’s function structure andcomponent module, identify the best methods and strategies used in the process ofimproving the former program with distributed parallel: choose the space divisionmethod which has the best expansibility to divide the whole calculation task amongthe three mainstream molecular dynamic parallel algorithms, on the basis of applyingOpenMP multi-core parallel acceleration technology, use the message passing modelthat is more suitable for the distributed environment and the characteristics of discreteelement method to form MPI+OpenMP two level’s mixture parallel structure; using amixture of peer to peer and master-slave MPI parallel programming design pattern,we can take advantage of all nodes’ calculation capability, as well as the master node can play the dominant role of a manager in the whole calculation process.Then weintroduce the customized MPI composite data passing types of different kinds ofboundary and particle classes. We decide to use blocking communication to finish allpeer to peer communications to keep the accuracy of discrete element methodsimulation, after analyzing blocking communication’s characteristic that calculationand communication don’t overlap each other. To solve the problem that the clustermay be in the state of deadlock or waiting when multi-node processes communicatewith each other at the same time, we employ the odd and even number nodecommunication dispatching method. This paper also talks about the improvement ofsome distributed DEM parallel algorithms that are all based on grid method:simulation space division algorithm, particle generating algorithm, neighborhoodsearch algorithm, distributed load balancing algorithm and distributed file writing andsummarizing method etc. On the basis of not changing AgriDEM program’s logic andfunctions, revise the previous algorithms and data structures pertinently according tothe new algorithms above, in order to make the new program perform DEMsimulation tasks normally in the environment of distributed computing cluster system.Finally we have tested the distributed parallel program in different simulation scenesand prove the feasibility and effectiveness of three-dimensional DEM distributedparallel computing.