Research On Optimization Method Of Reaction Force Field Parameters Based On Reinforcement Learning

Reactive Force Field(ReaxFF)based on molecular dynamics simulation has been playing a very important role in computational materials science,which mainly uses computers to simulate chemical reactions under very harsh conditions to study the physical and chemical properties of elements in the system.The quality of the ReaxFF will directly affect the accuracy of the final property conclusion,therefore,how to obtain high quality ReaxFF has become a significant research topic in the field of computational materials.Due to the form of ReaxFF is complex and difficult to determine so the optimal parameters of ReaxFF can only be obtained by parameterizing and optimizing the parameters,which requires more computational cost and seriously affects the quality of the final ReaxFF.Aiming at the problems existing in the current optimization method of the parameters of ReaxFF,this thesis summarizes and analyses the related research status at home and broad and conducts research on the optimization method of ReaxFF parameters based on reinforcement learning.Firstly,in this thesis a parameters optimization system is designed through studying the relationship between reinforcement learning and parameters optimization of ReaxFF.The system mainly includes the design of distributed storage system of ReaxFF data files and reinforcement learning model,in which the current value,historical gradients and reward are taken as states that ensure the invariance of displacement.Discrete and continuous schemes are designed in action space and reward function are designed as continuous.It also includes the design of training algorithms and the evaluation module of reinforcement learning model.In the algorithm design of reinforcement learning model training,Deep Q Network algorithm in discrete action space is studied in which the problem of mutation point of traditional DQN algorithm is solved by introducing attention mechanism.This algorithm successfully applies the trained model parameters to similar tasks to improve the efficiency.In order to reduce the dimension of action space and solve the problem of initial decoupling,the algorithm is improved via using simulated annealing method on the basis of the research of Asynchronous Advantage Actor-Critic algorithm in continuous action space where thirty-two agents are used to interact with the environment at the same time under different strategies.The truncated Gauss distribution with discount factor is used to reinitialize the state when reaching the terminal state and the core agent comprehensively analyses and learns the experience of each agent to obtain the optimal strategy.The algorithm solves the problem of initial decoupling of reinforcement learning and realizes parallel search,thus improving search efficiency and accelerating convergence of search.A reinforcement learning algorithm based on model learning and Monte Carlo Tree Search(MCTS)is proposed in this thesis for the sake of further improvement of accuracy of ReaxFF.In this algorithm the data produced during operation of the previous reinforcement learning algorithms are used to learn the state transition model of the environment of reinforcement learning and the reward function.The combination of the model and MCTS further improves the efficiency and accuracy.Finally,the efficiency and accuracy comparison experiments and parameter translation experiments are designed for the above algorithms.The algorithms proposed by this thesis are more efficient,adaptive and reusable than GARFfield which is proved by the results of these experiments.The final force field parameters are applied to the specific chemical reaction simulation to verify the accuracy of the force field parameters,which meets the expectation of this thesis for the reinforcement learning method of force field parameters optimization.Comparing the reinforcement learning algorithms with the gradient-based optimization algorithms in the optimization task of fluorine where the results show that the reinforcement learning algorithms can be applied to the optimization domain.