Research On Autonomous Cage Inspection Of Marine Ranch Based On Reinforcement Learning

Since the 21 st century,with the continuous improvement of China’s economy,the country has begun to develop and utilize marine resources in a reasonable way.Among them,net cage aquaculture is an important part of utilizing marine resources.As marine pastures are mostly located in deep sea,net cages used for aquaculture are built under the deep sea with complex environment,which is prone to problems such as net damage,corrosion,and fish escape.Therefore,regular net cage inspections are necessary in the process of net cage aquaculture.Using underwater robots for dynamic inspections is one of the important development trends in the future.This article focuses on the issue of autonomous net cage inspection by underwater robots and carries out related research,mainly completing the following tasks:(1)The problem of autonomous net cage inspection is transformed into a visual or acousticbased underwater robot net cage inspection problem.Reinforcement learning algorithms are used to solve the net cage inspection task.Because reinforcement learning learns strategies by continuously interacting with the environment,accumulating rewards and trial-and-error is too costly in the real environment.Therefore,a simulation environment is used for training.First,the aquaculture net cage is modeled.Secondly,the underwater robot’s linear velocity,angular velocity,Euler angles,etc.are modeled.Finally,sensor parameters,work modes,and motion control are modeled to make subsequent training more convenient and effective.(2)Propose a self-learning underwater robot net cage inspection strategy based on SAC-AE combined with vision.In this paper,the net cage inspection problem is modeled as a continuous action Markov decision process(MDP).The camera-acquired image is used as the state,and the action is defined as the control of the AUV’s linear and angular velocity.A reward function is designed based on factors such as the AUV’s offset distance,heading,and operating speed.The neural network structure of the policy is designed,and SAC-AE is used for policy learning.The Auto-Encoder(AE)can effectively extract low-dimensional latent features from highdimensional data,effectively solving the problem of SAC being unable to train with raw images and falling into local optima.The simulation experiment section demonstrates that SAC-AE has advantages over SAC.It is proven that this method can effectively solve the problem of underwater net cage inspection,and has good applicability and practical value.(3)A method for autonomous net-cage inspection of underwater robots based on the PPO algorithm combined with acoustics is proposed.The net-cage inspection problem is also modeled as a continuous action MDP.The camera on the AUV is replaced with a sidescan sonar,and the minimum distance,distance difference on both sides,and the linear and angular velocities of the AUV are extracted and calculated as the perception input.A multi-constraint reward function is designed based on factors such as heading,deviation distance,and operating speed.The PPO algorithm is used to learn the optimal net-cage inspection control strategy,which is compared with the SAC algorithm,and the experiment shows that the PPO algorithm is more suitable for this method.The experiment results show that the AUV can learn an effective control strategy based on the actual net-cage mesh density.Finally,based on a real AUV inspecting a net-cage,the AUV is placed inside the net-cage for inspection using the original net-cage framework.The results show that an effective control strategy can still be obtained,demonstrating that the proposed method can obtain a control strategy for the AUV to complete autonomous net-cage inspection.