A Supplier Bidding Strategy Based On Q-Learning Model

The storm of power industry's reformation went around the world during the 80th of last century. In order to improve the efficiency of producing electricity, lower the price, optimize the configuration of various renouncements, break the leading structure of monopolization and reach the optimization of social benefit, the competition mechanism must be introduced by developing power market. Agent technology has been widely used recently in the field of studying and simulation on power market, because its performance in simulating, forecasting and dealing with the uncertainty and instability of power market proves to be better than traditional methods.This thesis focuses on the problem of maximizing the profit of the supplier Agent in a day-ahead power market. The supplier's bidding strategy is modeled by Q-learning algorithm. It is used to simulate tacit collusion between generation companies too. The modeling process of using the algorithm in power market's simulation is discussed. The advantage of using this reinforcement learning method to help the supplier making bidding strategy is introduced by simulation. It's discovered that the supplier can make a complete bidding strategy to maximize its daily profit using Q-learning. Considering the congestion of network and ramp rate, the supplier still can make accurate judgment to the state of power market which leads them to change its bidding strategy. The result of simulation shows that if we increase the number of suppliers who use Q-learning, the market clearing price will be higher. Besides, another Q-learning model established in this thesis is able to simulate the tacit collusion between low cost generations which make the profit of high cost generations be less. It's found that the most efficient way to avoid collusion is to lower the market share of every generation and increase the elasticity of demand.