| With the increasing development and recognition of the sea,the visual underwater vehicle has becoming the equipment of crucial importance for sea exploration,which are widely used in underwater search-and-rescue andsurvey and biological monitoring.Visual environment perception,acquiring,analyzing and recognizing environment,is a key step for visual underwater vehicles,which handles and feedback visual information to assist the autonomous decision making,such that the correctness and of decision and the safety of underwater vehicles are guaranteed.The real time for environment perception and autonomous control decision differs in various underwater tasks.The unmatched perception and control strategies might cause the asynchronization of state extraction and control decision and yields weak robustness of autonomous control for underwater vehicles.Even more severely,the control system might be unstable and then lead to task failure.In this context,this paper aims to solve the autonomous reinforcement control decision of underwater vehicles with the perceptions of discrete vision(image)and continuous vision(video),which utilizes deep learning to extract perceptual features via environmental,data driven and use reinforcement learning to obtain action decision via sailing state driven,such that the requirements of autonomous perception and analysis,and decision-making and control can be satisfied.The main researches are summarized as follows:Based on the development and research actuality of unmanned underwater vehicle and its intelligent perception and decision-making algorithms,the theories of deep reinforcement learning are introduced.For the depth prediction of discrete vision,a fully convolutional residual networks,adopting the architecture of encoder-decoder and residual learning,trained with supervised learning,is proposed for the extraction of discrete depth information.It solves the blurred edges and missing details of depth prediction of discrete vision in underwater environments.Considering the vision displacement between frames in a continuous visual perception(video),based on image depth extraction,by introducing a ego motion estimation network for matching between adjacent two frames,a disparity network is proposed for the extraction of continuous depth information.It solves the problem of depth prediction delay for continuous vision of underwater environment.With the discrete visual perception,under the Q-learning reinforcement learning control framework,by using convolutional neural networks for feature extraction from discrete depth image,an autonomous control network D3QN is proposed for discrete autonomous decision-making control using the mechanisms of greedy competitive learning and online training.For the autonomous decision-making control with continuous visual perception,under the Actor-Critic reinforcement learning control framework,by utilizing convolutional neural networks to simulate the calculation of strategy and Q value from the state inputs,i.e.,continuous depth video,an autonomous control network D3PG is proposed for continuous control using the the online training mechanism of Q-learning.By incorporating the discrete and continuous visual perception and corresponding autonomous reinforcement control methods,the joint simulation training and experiments are conducted.The experimental results demonstrate that the proposed autonomous reinforcement control method can realize the autonomous control and obstacle-avoiding navigation of unmanned underwater vehicles with visual perception and yields high accuracy and strong robustness.This also verifies the effectiveness and usability of the proposed methods which might provide a new fashion for the application of unmanned underwater vehicles in shallow waters. |
