Research On GrHDP Algorithm And It’s Application On Virtual Interaction Environment

In recent years, with the development of computer science technology, whether thecomputer could be designed in accordance with human thinking and learning abilitybecomes a hot topic in academic world. So far, the academic’s solution of this topicfocuses on discussion about how to develop a signal processing mechanism to imitate thebiological brain organisms for closing the gap between human behavior and computerdecisions. The intelligence and adaptability of brain-like mechanism has bright prospect invarious fields, such as optimal control, aerospace, information processing, intelligent robotindustrial applications, and various others. It has been an important developing trend offuture society and science technology.In this paper, a new adaptive dynamic programming (ADP) algorithm, called goalrepresentation heuristic dynamic programming (GrHDP) algorithm, has been studied as astarting point to investigate algorithm solution of mechanism design about brain-likedecision processing. Besides, virtual reality (VR) technology has been integrated into ADPalgorithm research. A VR interactive simulation platform based on learning and controlsystem (VR-GrHDP algorithm platform) has been developed, where the platform could beused to verify the learning ability of GrHDP algorithm applying on different problems.The details about this paper’s works are as follows:The theoretical background of GrHDP algorithm, including ADP algorithm, Markovdecision process (MDP), dynamic programming (DP), and reinforcement learning (RL),have been analyzed. Based on these basic theories, the principles of GrHDP algorithm,including the architecture, learning process, weights updating process of neural networks(NNs), have been intensive studied. The optimal decision design of GrHDP approach todifferent learning and control systems have been discussed, such as nonlinear controlsystem, adaptive learning system, optimal control system etc.The synthetic structure, modeling and interaction methods of virtual reality modelinglanguage (VRML) have been studied. Based on VRML, a VR interactive platform basedon learning and control systems, called VR-GrHDP algorithm platform, has been developed. The platform approaches to virtual environment simulation experiment inMatlab through Simulink3D animation interface. The learning ability of GrHDP approachcould be demonstrated in different virtual experiment scenarios, while the reliability andstability of GrHDP approach to learning and control problems could be verified throughexternal virtual events creating.Two virtual interactive environments of nonlinear control systems based ontriple-link inverted pendulum balancing system and ball and beam balancing system aredeveloped. The reliability and stability of GrHDP algorithm on nonlinear control problemshave been verified through virtual interactive experinments. The simulation resultsdemonstrated two aspects. First, GrHDP algorithm has strong robustness and stability onnonlinear control problems in both situations with and without external disturbance.Second, the virtual interactive platform provides an intuitive, convenient and interactiveinterface for nonlinear control system algorithm research.The robot navigation strategy design based on GrHDP algorithm has been proposed.The solution of GrHDP approach to three navigation problems are discussed, where arethe discrete path navigation, continuous path navigation and goal seeking navigation. Themobility, flexibility and adaptively of GrHDP algorithm response to different environmentand different obstacles’ situations have been verified through different virtual interactiveevents. The simulation results show that:(1) the learning efficiency and square error ofGrHDP algorithm are much better than HDP algorithm and Q-learning algorithm in mazenavigation problem;(2) the reliability and stability of GrHDP algorithm are much betterthan HDP algorithm in different track path navigation problem;(3) GrHDP algorithmprovides the an effective mobility, adaptively and flexibility in goal seeking problem evenwith different start situations or different goal’s situations.The industrial application of VR and GrHDP algorithm has been discussed, where theresearch object is the coal transportation operation system of Qinhuangdao port. The3Dvisual monitoring control system of coal transportation operation process has beendesigned, and the optimal control solution to motor speed of car dumper has beendiscussed. On one hand, the multi-dimensional monitoring data information could beinstead of one dimensional data information by using VR technology, network technology and sensor technology. On other hand, the comparison results demonstrate that in optimalcontrol problem of motor speed of car dumper, the response speed and fitting degree ofGrHDP algorithm are much better than classic PID control and RBFNN-PID controlmethod.