| As the small-scale unmanned helicopter has special properties,such asVTOL(Vertical Take-Off and Landing),agile flying ability,large load capacity,it plays an important role in both military and civil applications.But it faces some challenges in automatic control because of it’s strong nonlinearity,dynamic coupling and hiding states.Thus,in many practical operations,human pilots still need to control these helicopters.On the other hand,with more and more complicated environment,an effective way of avoiding blocks and route planning has became an major task.This thesis focuses on the developing of attitude control,height control and navigation for a small-scale helicopter,based on the reinforcement learning control with deep learning.Some flight experiments are presented to illustrate the effect.Firstly,this thesis uses a method of reinforcement learning—Q learning as the navigation algorithm.But the Q-learning based on table always has many states that will introduce the so-called“dimension disaster”issue.So this thesis utilizes the deeplearning networks to handle states problem as an improvement,with neural networks to replace the Q value.To train network parameters,a simulation of interaction between unmanned helicopter and environment is put into practice.As training going on,the task to find the optimal trace can be completed.Secondly,this thesis divides the helicopter model into two parts,including nominal model and uncertainty model,to compensate the influence of the latter.A continuous nonlinear robust control strategy is presented point to the nominal model,reducing the disturb of wind.while an online reinforcement learning control algorithm is used to compensate uncertainty model.Semi-global asymptotic stability of the error signals of the helicopter’s attitude is proved via Lyapunov based analysis.Finally,real time experiments are performed on the helicopter testbed.These results prove that the control method proposed from this thesis can reach good performance in helicopter’s robustness with uncertainties and disturbances.Lastly,this thesis designs an attitude control strategy with offline reinforcement learning.The method first uses prior data to establish a Local Weight Linear Regression model.Then a random approximation iterative method is adapted to train the model,in order to optimize the controller parameters.The convergence of this iterative method is proved by probability based analysis.In the end,by put the training parameters into real flight experiments,thesis gets the conclusion that the height control error and the parameters adjusting time is smaller than manual adjustment,the results validate the good performance of the proposed scheme. |
