An Adaptive Tracking Control Method For Wellbore Trajectory Based On Reinforcement Learning

The wellbore trajectory tracking control refers to controlling the actual drilling direction according to the pre-designed wellbore trajectory to reach the target position in the stratum.However,in the actual construction process,wellbore trajectory tracking control is difficult,and there are often many challenges such as strong interference,high coupling,and time-varying.However,most traditional wellbore trajectory tracking control methods are usually constructed based on certain constraints or assumptions,which have limited ability to reflect the actual drilling process.Moreover,these methods have low intelligence,poor anti-interference,and poor adaptive ability.Therefore,this thesis proposes an adaptive tracking control method of borehole trajectory based on reinforcement learning to address the above problems.This method adopts the deep deterministic policy gradient(DDPG)model based on the priority experience replay mechanism,and accelerates learning through transfer learning,thereby constructing a wellbore trajectory adaptive tracking control system with anti-interference ability.The main work of this thesis is as follows:(1)A 3D simulated drilling environment based on the uncertain interference mechanism is designed for training and testing of the wellbore trajectory tracking control method.The system uses generative adversarial networks to expand the geological profile and geological feature data,and uses the geological profile interpolation method to complete the geological modeling work.At the same time,through the geologicalengineering high interaction mechanism and the uncertain interference mechanism,the system provides a good platform for the adaptive training of the subsequent wellbore trajectory tracking control method;(2)A wellbore trajectory tracking control algorithm based on DDPG is established.The algorithm defines wellbore trajectory control action,state space and reward function in detail,and designs a priority-based experience replay mechanism.Experiments show that the algorithm can adjust the tool face angle according to the current well deviation data,and accurately complete the tracking of the preset trajectory.At the same time,experiments show that the priority-based experience replay mechanism effectively improves the training efficiency;(3)A wellbore trajectory adaptive tracking control method based on transfer learning is proposed.This method optimizes the original state space and reward function,and transfers the original policy network and value network to the new model through transfer learning,which improves the learning efficiency and generalization ability of the algorithm.Experiments show that the new model has good anti-interference performance in the 3D simulated drilling environment with uncertain interference,and can accurately guide the wellbore trajectory to the target area.At the same time,the adaptive ability of the new model has been greatly improved.It can complete the trajectory optimization according to the actual measured data while drilling,and improve the drilling encounter rate of the target oil layer.The adaptive tracking control method for wellbore trajectory based on reinforcement learning proposed in this thesis can complete the tracking of the preset wellbore trajectory in complex geological environment,reach the target area accurately,and show good antiinterference and adaptive ability.