Research Of Wind Farm Operating Strategy Based On Deep Reinforcement Learning In Electricity Market Environment

China has become the country with the largest installed wind power capacity in the world,and the wind power generation has become an indispensable means for adjusting the structure of the whole power generation.At the same time,a new round of power system reform is in full swing.The fair and equitable electricity market is gradually being improved.Wind farm as a wind power generator entering the electricity market to chase profits has become an inevitable trend.However,the uncertainty of wind power and electricity price brings challenges to the economic dispatch of the power system.In this context,how to coordinate the dual identities of wind power generation as a passive power source and the power generator,that is,how to cope with multiple uncertainties while chasing profits has become a significant problem for wind power generationEnergy storage systems(ESSs)and other reserves with regulation capabilities as the dispatching objects of wind farms can solve the above problems.The current optimization algorithms against uncertainty can be divided into two categories:one includes scenario-based stochastic optimization,dynamic planning,and opportunity constraints.These methods all require the uncertainties of wind power or electricity price to be assumed as a known mathematical probability distribution during the optimization.However,whether the assumed probability distribution matches the actual uncertainty laws will directly affect the optimization effect of the entire optimization algorithm.Robust optimization method as another type method utilizes an interval to consider the worst result of uncertainties.The artificial assumption of uncertainty can be avoided and the calculation efficiency is improved.However,this type of method fails to effectively mine the law of uncertainty,resulting in conservative decision-making results.It is questionable whether this type of optimization method can be adopted in the electricity market environment where revenue is the primary goal.This paper focuses on the wind power generation strategies based on deep reinforcement learning(DRL).DRL is a model-free optimization method based on big data.Uncertainty laws of wind power and electricity price can be captured completely by mining the big data.DRL gets rid of the limitations of traditional mathematical optimization methods.This paper describes continuous optimal control of wind generation as a Markov decision process,and points out that DRL can naturally deal with uncertainty.Then,a wind-ESS-reserve combined generation method based on DRL is proposed,which uses big data to directly drive the optimization.The optimization process does not require the artificial assumptions of the uncertainty.Both ESS and external reserve are considered as the means for regulation first time.The entire method is driven by big data,and the deep neural network is used to approximate the optimal control strategy against uncertainties.The DRL algorithm used is implemented on the Rainbow framework.Furthermore,this paper improves the traditional dispatch framework and proposes a schedule mode integrating wind power prediction and ESS decision,where high-dimensional wind farm states with raw measurement data directly drive the ESS.Compared with the traditional schedule mode where prediction and decision are separated,the new mode integrates wind power prediction and ESS action decision-making,avoiding the loss of effective decision-making information in the prediction stage.The data from Jiangsu Wind is used to finish case studies.The experimental results verify the effectiveness of DRL in coping with uncertainty and the superiority of the proposed integrated schedule model.