Decision Model Research On Driving Teaching Based On Cellular Automata And Reinforcement Learning

With society developing rapidly,the number of civilian vehicles is increasing and excellent driving technology has become a necessary skill for modern people.In order to improve the efficiency of driving school training,people began to attach importance to the development and use of robot coaches.However,the fixed teaching rules of the existing robot coaches can't be used for teaching innovation and teaching according to the aptitude of different trainees.It is difficult for robot coaches to achieve the teaching effect of real coaches.To use robot coaches to realize the function of teaching according to the aptitude,there are two main problems.One is that it is difficult to establish an accurate mathematical description model for the trainees because during the driving training,the aptitude of the trainees change greatly and the operation of the trainees is random.The other is that establishing a driving school coach decision-making model is difficult because of uncertain aptitude of the trainees.In response to the above two problems,this paper analyzes the driving training process:the process of driving training is decomposed into three parts:trainee feedforward,trainee feedback and coach feedback,whose impact on the driving trajectory is discussed separately.Then,this paper analyzes coach's teaching process,compares the difference between real driving coach and autonomous driving and discusses the advantages and disadvantages of the robot coach.Based on the above analysis,this paper obtains the following innovations by cellular automata and reinforcement learning algorithms:(1)Aiming at the problem that the characteristics of trainees change greatly and the operation has randomness,this paper takes reverse storage as an example,considers the trainee and the vehicle as a whole,and designs a cellular automaton model.Using hexagonal grids to divide the cell space,discretize the direction of cell movement,determine the state and neighbors of the cell.A new type of cell evolution rule is proposed considering the effects of trainee self-learning ability,trainee feedforward,trainee feedback,and coach feedback on the direction of vehicle movement.The simulation results show that the cell automata model designed by trainee and vehicle can simulate the driving trajectory of trainees with different characteristics.And the validity of the model is verified by using the driving data of real students.(2)In view of the problem that the driving school coach decision-making model is difficult to establish when the characteristics of the trainees are uncertain,this paper gives a solution to train the driving school coach decision-making model using reinforcement learning.Based on the cellular automaton model of trainee and vehicle,the Markov decision process quadruple is established,and the coach decision model is obtained by training with strategy iteration algorithm and Q-learning algorithm respectively.Training for different trainee and vehicle models can get different coach decision-making models,and then realize the function of teaching according to aptitude.The simulation results show that the coach models trained by both algorithms can play a good role in guiding students' practice.And the validity of the model is verified by using real data of the students under the guidance of a real coach.In summary,this paper analyzes coach's teaching process,designs a cellular automaton model of driving school trainee and vehicle and proposes a program that using reinforcement learning to realize the teaching function of robot coaches according to trainees' aptitude.The results help to make up for the defects of existing robot coaches and improve the training of robot coaches effectiveness.