Research On Feedback Stabilization Of Boolean Networks Based On Reinforcement Learning

Boolean networks,as a common model of biological networks and gene regulatory networks,has received extensive attention of researchers in the past decades.Considering genes(or proteins)and their regulatory relationships in living cells as nodes and logical relationships,respectively,gene regulatory networks can be modeled as Boolean networks.In Boolean networks,the logical relationship is the focus of research,which reveals the specific evolution process of the state of the genes,but most of the existing theoretical tools are difficult to express these logical relationships into a simple mathematical form.In this paper,the dynamics of Boolean networks is converted into a discrete system.With the help of the semi-tensor product of matrices,the mathematical expression of the discrete system is simplified,which is conducive to the direct application of the existing control theories and control methods to the research of Boolean networks.In actual biological networks,there usually exists external inputs,internal outputs and noise.Taking these into account,the dynamic evolution of gene regulatory networks can be analyzed more accurately.After the control inputs are introduced,we obtain Boolean control networks.By virtue of some control algorithms,the evolution of Boolean control networks can be guided.The main control algorithm adopted in this paper is the temporal-difference algorithm in reinforcement learning.In the second chapter of this paper,we first expound the basic framework and principle of reinforcement learning,and then introduce the classic temporal-difference algorithm.By virtue of algorithms in reinforcement learning,this paper studies the feedback control problem of Boolean networks.The specific research contents are as follows.There exists noise interference in gene regulatory networks,and these noises have the characteristics of stochasticity.After introducing these noises into Boolean control networks,a stochastic Boolean control network is obtained.The third chapter of this paper presents the dynamics of stochastic Boolean control networks and converts them into the algebraic form,and then studies the output tracking problem of stochastic Boolean control networks.Next,we introduce the Sarsa algorithm.The main goal of this chapter is to explore the model by the agent in reinforcement learning to find the optimal control input corresponding to each state node when the output is given,so that the system can quickly track the given output.The main implementation process is to convert the output tracking problem into a set stabilization problem firstly,and then convert it into a path optimization problem.The path optimization process is carried out by the Sarsa algorithm.Next,according to the optimal control sequence calculated by the Sarsa algorithm,an optimal state feedback controller is designed to solve the set stabilization problem.In addition to the stochasticity of the state transition of the entire network caused by noises,stochastic selection of a state node to update at each time instant will also cause stochasticity problems.This kind of Boolean control networks is called asynchronous Boolean control networks.The fourth chapter of this paper studies the output feedback control problem of asynchronous Boolean control networks.Firstly,the algebraic state space expression of asynchronous Boolean control networks is obtained by semi-tensor product,and on this basis,a sufficient and necessary condition for judging whether an asynchronous Boolean control network is asymptotically stabilizable in the infinite domain is proposed.Next,the Q-learning algorithm and its improved algorithm,double Q-learning,are introduced.Then,the relationship between output feedback and state feedback is analyzed,and a sufficient and necessary condition for the existence of output feedback controllers in an asynchronous Boolean control network are proposed.We then employ double Q-learning algorithm to design the output feedback controllers and elucidate why these controllers are optimal.