Research On Key Technologies Of Interactive Augmented Reality And Its Application In The Virtual Performance Of Qin Opera

Qin Opera is a treasure of the historical development of Chinese civilization and one of the oldest traditional operas.At present,the performance of Qin Opera is limited and the communication channels are single,which makes Qin Opera face the danger of inheritance and generation,or even disappearance.With the improvement of computer perception ability and the development of virtual reality technology,the interactive digital virtual performance of traditional opera realized through the application of technologies such as action recognition and augmented reality has become a research hotspot in the field of virtual reality.Aiming at the key links such as action recognition and 3D model registration in Qin Opera virtual performances,this thesis proposes an improved spatiotemporal graph convolution network model based on skeleton data and a pose estimation algorithm based on POSIT optimization,solved the problems of low accuracy of action recognition in traditional methods and insufficient accuracy and timeliness of camera pose estimation methods.Besides,a prototype system for virtual performance of Qin Opera was realized.The work I do is as follows:(1)A set of classical Qin Opera action datasets based on skeleton information is constructed.Use the device Azure Kinetic DK to collect the three-dimensional joint point coordinates of the human skeleton related to the action and organize it into a skeleton sequence dataset,which contains a total of 7 kinds of actions,and the sample size of each action is 120.(2)A human action recognition method based on fusion channel attention mechanism(SENet)improved spatiotemporal graph convolutional network(ST-GCN)is proposed.The spatiotemporal graph convolutional network establishes a space-time map to extract spatiotemporal features in two dimensions: space and time,and the channel attention module is integrated,so that the model can strengthen the attention to important features and weaken the influence of unimportant features.Observing the experimental results,we can find that the identification accuracy of the proposed method is significantly improved,and the top-1accuracy of action recognition on our Qin Cavity classical action dataset reaches 92.44%.(3)A novel camera pose solution algorithm based on POSIT optimization is proposed.First,the three-dimensional virtual control point is introduced in the world system,and then the two-dimensional pixel coordinates corresponding to the virtual control point are solved by using the camera model to form 3D-2D coordinate point pairs;Secondly,the POSIT algorithm and coordinate point pairs are used to estimate the relatively accurate camera posture;Finally,the Distance Between Control Points is optimized using the Gaussian Newton method to obtain a more accurate posture.Experimental results show that the proposed algorithm is more accurate than the DLT algorithm and the EPn P algorithm,and the operation time is shorter than that of the LHM algorithm.(4)A prototype system of Qin Opera theater virtual performance based on Unity3 D engine is constructed.The system includes action recognition,face detection,virtual model registration and other modules.The test results show that it has a strong sense of reality.