Research On Zero-Sum Game Approximate Optimal Control Of Modular And Reconfigurable Robot System Based On Value Iteration

As modern science and technology continue to advance,robot technology is also constantly improving,and the application field of robots is constantly expanding.People hope that robots can more flexibly adapt to various working environments and complete more complex tasks.Traditional robots are difficult to adapt to changes in working environments and tasks due to their own mechanical structure.If the robot is re developed due to changes in working environment and tasks,the investment is often huge and the development cycle is long.This means that there are certain work environments for which accurate predictions cannot be made beforehand.For example,in earthquake rescue,people can not accurately know the specific environmental conditions of the people to be rescued under the ruins.Therefore,a robot that can change its own structure according to the new working environment and new task requirements is required to complete the task.One way to solve this task is to develop a reconfigurable robot system.Reconfigurable robot is composed of some interchangeable robot modules with various geometric dimensions and functional characteristics,which can adapt to different work needs.Reconfigurable robot systems will inevitably work in unknown environments.It is important to prioritize stability,durability,and energy efficiency.Therefore,it is necessary to adopt appropriate control strategies under the conditions of uncertain environmental information.Optimal control research,an essential part of modern control theory,focuses on identifying the most suitable control strategy for a particular controlled system.The key objective is to optimize the system’s performance indicators by employing the chosen strategy.For reconfigurable robot that is disturbed by the external environment,the twoplayer zero-sum game theory is introduced into the robot system.The external disturbance term and the control output terms of the system form a game.To improve a reconfigurable robot’s control,we can approach it as a game between two players.One player represents the controller while the other is the disturbance term.In this game,both players strive to minimize disturbances while maximizing the controller’s effectiveness.We can determine the best control strategy by solving the Hamilton-Jacobi-Issacs equation.The equation being discussed falls into one of two categories: either it’s a nonlinear partial differential equation or a nonlinear difference equation,both of which are notoriously challenging to solve analytically.Due to this complexity,obtaining an analytical solution is often not feasible and requires the use of numerical methods for approximation.The iteration method in ADP mainly consists of two parts,including value iteration and policy iteration.It should be noted that the stability of the policy iteration process is influenced by the initial admissible control strategy,but the appropriate initial admissible control strategy is often difficult to obtain.The value iteration algorithm is not restricted by this condition.By selecting the semipositive definite function as the initial value function,it converges to the optimal value iteratively,by guaranteeing the stability of the entire control process of the system,we can enhance the control performance of the reconfigurable robot and minimize the energy consumption cost associated with the controller,based on the optimal control problem of the reconfigurable robot system,combined with two-player zero-sum game theory and value iteration algorithm,this paper conducts in-depth research on the optimal control problem of the reconfigurable robot system based on zero-sum game.(1)Dynamic modeling of reconfigurable robot systemsThe dynamic model of the entire reconfigurable robot system has been broken down into separate dynamic subsystems and modeled individually,taking into account the practical implications and generalizability of the model.We have developed dynamic models for the robot system using the Newton-Euler iterative algorithm and joint torque feedback method.We have also conducted a comprehensive analysis of modeling uncertainty,which provides a strong basis for implementing subsequent strategies such as decentralized control,zero-sum game methods,and compensating for model uncertainty.(2)Value iteration-based zero-sum neuro-optimal control of modular and reconfigurable robots via adaptive dynamic programming under external interferenceWe have developed a zero-sum neural optimal control method based on value iteration for the reconfigurable robot subsystem,using its constructed dynamics model.This method addresses the issue of inaccurate position and velocity trajectory tracking caused by external environmental factors that impact the reconfigurable robot’s performance.The adaptive fuzzy control method is used to identify Coriolis force,centripetal force and gravity term.The proposed value iteration algorithm starts with any positive semi-definite to ensure the stability of the system control process and ensure that the iterative value function converges to the optimal solution and the convergence analysis is given.The value iterative-based algorithm uses neural networks to approximate the Hamilton-Jacobi-Issacs equation and obtains the decentralized optimal tracking control strategy.The simulation results of reconfigurable robots with different configurations show that the proposed control method is effective.(3)Event-triggered adaptive fuzzy optimal control of modular and reconfigurable robots using zero-sum differential game through value iterationWe have proposed an event-triggered zero-sum neural optimal control method for the reconfigurable robot system,using its dynamic model based on joint torque feedback.By implementing value iteration,we have successfully solved the position and velocity tracking problem of the robot.Firstly,we have developed a controller that makes use of a fuzzy control algorithm to address the uncertainties present in the dynamics model of the subsystem.Secondly,the disturbance of the unknown environment and the optimal control of the robot are considered as two participants in a zero-sum differential game,and the challenge of ensuring stable control for a reconfigurable robot system that operates in an unfamiliar environment can be restated as an issue of optimal control.On the basis of eventtriggered mechanism and value iteration algorithm,the event-triggered Hamilton-JacobiIssacs equation is approximately solved by establishment of actor-critic structure.As an effective technique for achieving optimal control of complex nonlinear systems,iterative algorithm has an incomparable advantage in optimal control of reconfigurable robot systems.In contrast to conventional adaptive dynamic programming algorithms,there are notable distinctions in the approach taken by this approach,the value iteration algorithm is introduced in this scheme,which overcomes the initial condition restriction of policy iteration.By selecting the semi-positive definite function as the initial value function,the convergence to the optimal value through iteration can ensure the stability of the whole control process of the system.Moreover,according to the Lyapunov stability theorem,it has been demonstrated that the tracking errors of a closed-loop system remain within an acceptable,bounded range over time..At last,experimental examples and comparisons show the reliability and effectiveness of this method.