Equilibrium Analysis Methodand Its Applications For Electricity Market Using Reinforcement Learning

With the deepening of electricity market reform and the advancement of the construction of a new power system with new energy as the main body,the electricity market will present many new features,such as diversification of market entities,diversification of transaction varieties,high frequency of organizational timing and intensified competition risks.The electricity market mechanism design will face more complex problems.In order to solve the key coordination mechanism problems in the construction of China’s electricity market,the theoretical method of the equilibrium analysis needs to be improved and perfected,so as to provide a quantitative analysis tool for evaluating the effectiveness of the mechanism design.Therefore,aiming at the practicability defects of the existing methods,combined with the advantages of reinforcement learning,this paper conducts in-depth research on the equilibrium model and solution method of electricity market.Firstly,the existing equilibrium analysis methods do not consider the no-load cost and start-up cost of the unit,nor do they consider operating constraints such as minimum running time and minimum downtime.In order to make up for these deficiencies,this paper constructs a bi-level equilibrium model of electricity spot market considering the non-convex cost of units and the constraints of integer variables.By introducing the multi-agent(deep)reinforcement learning method,the iterative solution of the bi-level equilibrium model is realized.And the existence of the Nash equilibrium solution in the equilibrium model is further discussed,it proposes the judgment method of Nash equilibrium solution.Secondly,for the question of how to choose the pricing mechanism of the electricity spot market,on the basis of considering the collaborative process between the day-ahead market and the real-time market,this paper constructs a two-stage equilibrium model of the electricity spot market for three pricing mechanisms:system marginal price,zonal marginal price and locational marginal price.And use multi-agent deep reinforcement learning(MADRL)method to solve it iteratively.Combined with the basic theory of the pricing mechanism and the market equilibrium results,the applicable conditions of the three pricing mechanisms are theoretically demonstrated.Thirdly,aiming at the effectiveness evaluation of flexible resource incentive mechanism in the scenario of high proportion of renewable energy,this paper considers the coordination mechanism between electricity and new auxiliary service products,constructs bidding decision models including wind,solar,thermal power,pumped storage and other diversified market entities,and combines the joint clearing model of electricity and flexible ramping products(FRP).And a solution method based on MADRL,an equilibrium analysis method for the electricity spot market considering the joint optimization of electricity and FRP is proposed.By constructing examples,the functions of various flexible resources and the new auxiliary service products are studied.Finally,in view of the effective connection between the contracts for difference(CfD)and spot market faced by the construction of China’s electricity market,this paper establishes a mathematical model of market-based CfD,and proposes a method to determine the curve of government-authorized CfD based on the market equilibrium results.Combined with the generator’s bidding decision model considering risk preference and the multi-scenario spot market clearing model,an electricity market equilibrium model considering CfD and risk preference is constructed.And a MADRL algorithm based on risk management is improved to achieve an efficient solution to the equilibrium model.By constructing examples,the influence of two types of CfD and generators’ risk preference orr the market equilibrium is studied,and the rationality of the curve decomposition method of government-authorized CfD proposed in this paper is verified.