CO₂ Flooding Rates Dynaminc Optimization Based On Deep Reinforcement Learning

Low permeability and tight reservoirs are important succeeding reserves.CO2 flooding is an effective means to develop low permeability tight reservoirs.Multi-stage fracturing horizontal well CO2 huff and puff in tight reservoirs and multi-well CO2 continuous flooding in low permeability reservoirs are common development methods.Injection-production rates optimization has the advantages of easy implement,low cost and obvious effect.At present,the mechanism of CO2 flooding rates optimization is still to be further studied,and the existing rates optimization methods are insufficient.Therefore,it is of great significance to further clarify the mechanism of rates optimization and improve the deficiencies of existing rates optimization methods.First,considering the characteristics of deep reinforcement learning algorithms and the main characteristics of CO2 flooding rates optimization in different development modes,we selected the matching optimization algorithm and demonstrated its feasibility.The proximal policy optimization(PPO)algorithm is easy to converge and has high stability.It is suitable for CO2 huff and puff rates optimization of multi-stage fracturing horizontal wells.The injection and production parameters vary greatly with strong cyclic for CO2 huff and puff.The multi-agent deep deterministic policy gradient(MADDPG)algorithm considers the interference between multi-agent and the instability of the learning environment,and is applicable to the multi-well CO2 flooding rates optimization.Then,the new rates optimization methods based on the PPO and the MADDPG algorithms for the CO2 huff and puff with multi-stage fracturing horizontal wells and multi-well CO2 continuous flooding were established respectively.By comprehensively considering the optimization algorithm mechanism and the physical problems of CO2 flooding rates optimization,it is determined that the net present value is the optimization goal,the injection production policy is the optimization decision,and the constraints are clear.The mechanism model of rates optimization was established,and the effectiveness and stability of the new methods were demonstrated.Further analyzing of the rates optimization results and oil enhancement mechanisms,it shows that the development degree of CO2 huff and puff with multi-stage fracturing horizontal wells varies significantly in different reservoir reconstruction zones.The recovery efficiency of the main fracture and secondary fracture is high to 32%,which is far higher than the surrounding transitional zone reservoir matrix 6%,and the minimum is 1% in the far well unmodified area.Combined with the distribution of CO2 saturation,it is concluded that the main scope of CO2 huff and puff is the reservoir matrix in the fracture network reconstruction area and transition area.The main oil displacement mechanisms in each reservoir reconstruction area are different.The main oil displacement mechanisms in main fracture area,secondary fracture area and reservoir matrix transition zone are immiscible displacement and dissolved gas displacement;The main oil displacement mechanism in the unconstructed area of the far well is solution gas displacement;The contribution of depletion drive is negligible.The reason why the net present value of the optimized case is higher than the base case for CO2 huff and puff is that the difference in income from cumulative oil production is very small,and reducing the cumulative gas injection greatly reduces the gas injection cost.There is little difference between the two schemes in terms of cumulative oil production.Because the favorable impact brought by the full use of the reverse drainage stage is offset by the adverse impact brought by the reduction of cumulative gas injection and formation damage caused by stress sensitivity.The rates optimization of multi well CO2 continuous flooding can obtain the optimal net present value through two ways,expanding the sweep efficiency to improve the income from the increased cumulative oil production,and reduceing the ineffective injection to reduce the gas injection cost.For strong interlayer heterogeneity reservoirs with serious interference between layers,the general rates optimization effect of multi-well CO2 flooding is weakened,so layered injection production rates optimization is required.Finally,the new injection production rates optimization methods based on the deep reinforcement learning were applied to the actual blocks of Yuan 284 and Xinli 228.Verifing the effectiveness of the new methods,analyzing the rates optimization results and the oil enhancement mechanism,and providing guidance for the production on site.