Research On Hartmann Sensor Calibration Optimization Technology Based On Deep Reinforcement Learning

The Shack-Hartmann sensor is a common wavefront detector.the working.Firstly,incident wavefront is sampled and an image of all far-field distribution in all subapetures is got;then the slope vector is got by centroid calculation algorithm minus calibration vector;finally,the calculating wavefront is got through slope vector.In order to calculate an accurate wavefront,on the one hand,it's necessary to select a suitable centroid algorithm;on the other hand,it is necessary to do a calibration with an approximately ideal light source.For some optical systems,such as the adaptive optics system,the calibration result is equivalent to the ideal wavefront,which determines the upper limit of the system's correction ability.Thus,three problems have to be faced with: 1.How to choose an appropriate centroid calculation method;2.How to deal with the inaccurate calibration result caused by a non-idea calibration light source;3.The calibration result only contains the error of the wavefront detection part,and does not include the remaining errors in the system.Therefore,when the calibration result is not ideal,the system correction ability is greatly affected.In order to solve the above problems in the Shack-Hartmann sensor,this paper proposes a new solution.In the adaptive optics system,to get the best calibration result for the whole system,firstly,all of the errors are regarded as a set of liner mapping and nonliner mapping;then building an end-to-end optimal policy;finally,the calibration result is updated with the correction residual aberration under the optimal policy,and a new calibration result for the whole system is obtained.This solution frees up the work in physical process modeling and error analysis,and it's unnecessary to select an idea calibration light source.Based on this new idea,this paper proposes a Shack-Hartmann sensor's calibration optimization method based on deep reinforcement learning,and has achieved a series of innovative results.1.In this thesis,it is verified that the state transition process of the adaptive optics system satisfied the Markov property,and the aberration compensation process belongs to the Markov Decision Process.2.It is further verified that the problem can be solved with deterministic policy method.In the system,all the errors can be fixed as a set of a linear mapping and a nonlinear mapping.To optimize this set,an end-to-end method is used based on deep reinforcement learning.Firstly,a policy network and a value function network are builded up under the Actor-Critic structure;then,the optimization method about gradient transition is determined;finally,a set of suitable parameters for the adaptive optics system is determined.3.Focused on the characteristics of high-dimensional continuous output in the adaptive optics system,a Gaussian exploration strategy with attenuation term is proposed and compared with the traditional OU exploration strategy.Then,based on the beam metrics,such as Strehl ratio, factor and image sharpness,three reward functions are designed considering the uniform value range and updating gradient direction.Finally,simulation work is completed.4.To improving the data utilization rate,a random k-round TD-n method is proposed,which improves the data utilization rate by 1.5.Then inorder to solve the problem of slow convergence,two solutions,a pre-train method based on the supervised learning and a double reply buffer,are proposed.Finally,the calibration optimization solution is verified in the experiment.In this thesis,it is proved that in the adaptive optics,an end-to-end calibration optimization method based on the deep reinforcement learning is feasible.It also provides a solution for the problems satisfying the Markov Decision Process in other optics sytems.