Research On Artificial Emotion Model Of Virtual Doll Based On Q-Learning

In recent years,artificial emotion technology has gone through a great degree of development,and made fruitful research results,but it is far away from the artificial emotion technology being used by ordinary people.For the practical application of artificial emotion technology and Q-learning algorithm(a kind of reinforcement learning algorithm),this thesis mainly studies the construction of virtual doll emotion model and applies Q-learning algorithm into it on the foundation of previous research work.The traditional Q-learning algorithm has the problem that the reward model is simple and data transfer lag,and the layered emotion model based on cognitive evaluation has the problem of lacking of accuracy in the process of reasoning,so they cannot be applied to the virtual doll emotion model well.In order to solve the above problems,this thesis improves the layered emotion model based on cognitive evaluation and Q-learning algorithm,thus improves the ability of emotional reasoning and decision making of virtual doll.The details of the work are as follows:(1)In order to solve the problem that the reward model is simple and data transfer lag,this thesis has improved the traditional Q-learning algorithm.First of all,the efficiency of current learning cycle can be judged based on the relationship between current learning cycle step size and the historical smallest step size,then specify different reward strategies.The learning mechanism of update the Q value for each step is changed to update Q value in reverse after reaching the target state,an improved state-chain inverse stack Q-learning algorithm is obtained.The experiments show that the algorithm converges faster than the traditional Q-learning algorithm,but it is easier to fall into local optimum.According to the characteristics of the state-chain inverse stack Q-algorithm and the traditional Q learning algorithm,an improved hybrid state-chain inverse stack Q-learning algorithm is proposed,The first learning cycle of hybrid state-chain inverse stack Q-learning algorithm uses the learning mechanism of state-chain inverse stack Q-learning algorithm,and then uses the learning mechanism of traditional Q-learning algorithm.The hybrid state-chain inverse stack Q-learning algorithm is tested,the experimental results show that the improved Q-learning algorithm improves the learning efficiency and learning effect of decision making,and the validity of the algorithm is verified.(2)On the basis of the existing layered emotion model,in order to solve the problem of insufficient accuracy of fuzzy reasoning in layered emotion model,this thesis improves the method of cognitive evaluation.The evaluation of external stimuli was carried out from five dimensions(Desirability,Expectedness,Outcome Probability,Suddenness,Cause Harm),and a new evaluation rule table was designed.In order to solve the problem of the calculation method of emotion is rough in PAD space,this thesis uses a new calculation method,which takes into account the influence of emotional data variance on calculation of emotion distance.Then the simulation experiment has carried out,experimental results show that the improved cognitive evaluation method and the emotion calculation method used improve the emotional reasoning accuracy of the layered emotion model.(3)Based on the improved layered emotion model and hybrid state-chain inverse stack Q-learning algorithm,the virtual doll artificial emotion model based on Q-learning algorithm is designed.Firstly,the overall framework of the virtual doll emotion model is designed,and then the solution of problem such as the mapping of personality to mood in emotional reasoning system is given,and then put forward specific plan for the doll emotional decision system such as system frame design and behavior selection strategy and so on.Finally,the simulation experiment of the virtual doll artificial emotion model based on Q-learning algorithm is carried out.The simulation results show that the improved algorithm and the layered emotion model can improve the ability of emotional reasoning and emotional decision-making of virtual doll.