Research On Risley Prism Control Technology Based On Reinforcement Learning

Risley Prism is a beam deflection system used to change the beam propagation direction,adjust the beam direction and change the image axis.Its main principle is to achieve the purpose of changing the propagation direction of light beam through two co-axial prisms,prism 1 and prism 2,which rotate independently and freely.Compared with the traditional beam pointing system such as FSM(Fast Steering Mirror)and Gimbal,Risley Prism has obvious advantages.With compact structure and good dynamic performance,it can combine large scanning range,high pointing accuracy and high response frequency.And there is a strong possibility to replace the traditional beam pointing system,especially the Gimbal.It has a broad development prospect and is a hot research direction in the field of beam control.The closed-loop control of Risley Prism system is one of the focuses in the research of beam control.The main content of it is that a prism system captures the target to close loop,and then by calculating two prism rotating position of the prism system,and through designing and optimizing algorithm to control the rotation of the two prisms,rotating the two prisms to the calculated target position,so that the target image is always near the center of the detector’s field of view.It has a wide range of applications in target tracking,target aiming and other fields.However,compared with the traditional beam pointing system such as FSM and Gimbal,it also has obvious disadvantages in these fields.The closed-loop control of the traditional beam pointing system is more intuitive(For example,in the application of closed-loop tracking,the azimuth and pitch angles of the universal frame are easily decoupled,while the X and Y directions of the fast mirror are naturally decoupled).However,there is a nonlinear and strong coupling relationship between the miss distance,target position and prism rotation angle of Risley Prism,which leads to the complex calculation process in the closed-loop tracking application and hinders further research and application.In order to solve the above problems in closed-loop control of Risley Prism,this paper proposes a new solution.Based on that the rotation angle combination of two prisms with Risley Prism has only one corresponding relation of the outgoing direction of the beam(The outgoing direction of a beam is matched by two sets of prismatic rotation angles),this paper uses reinforcement learning.With the help of the miss distance information of the detector,the closed-loop control is carried out after setting the appropriate reward function.after giving a clear target,the natural self-learning ability is to applied to learn the optimal strategy to make the beam loop in the center of the detector.Firstly,in order to calculating of the orientation solution of Risley Prism beam,the analytical solutions of the first-order near-axis approximation method,the non-near-axis ray tracing method and the ‘coordinate transformation + geometry method’ are derived respectively,the difference and application range of first-order approximate near-axis and non-near-axis ray tracing methods are compared,and the beam direction control precision of Risley Prism is analyzed.Secondly,the feasibility of the application of reinforcement learning theory in the closed-loop control of Risley Prism is studied.Markov series theory is a powerful mathematical tool to solve the problem of time sequence.However,in the field of beam pointing control system,there has been no in-depth research.This paper verifies that the state transition process of the Risley Prism system accords with Markov property and that the closed-loop control process of Risley Prism belongs to Markov decision process.The optimization method of reinforcement learning for Risley Prism is also studied in this paper.Finally,according to the closed loop control characteristics of Risley Prism which determines that it belongs to the category of deterministic strategy,this paper determines the Actor-Critic network architecture,sets up the strategy network,action value network,the action space,state space and reward function,and establishes the simulation environment through Open AI Gym,which verifies the feasibility of the algorithm.Through the research in this paper,it is proved that it is feasible to use a closed-loop control method based on deep reinforcement learning to do miss distance in Risley Prism system,and a new solution is provided for other beam pointing control systems or other process optimization problems in the optical field that satisfy Markov property.