VR System Based On Spatiotemporal Semantic Mining Analysis And Implementation Of User Behavior

With the rapid growth of the virtual reality industry,the number of users participating in virtual reality has rapidly increased,and analyzing user behavior in virtual reality scenarios has become increasingly important.In current research,VR is often used as a means for researchers to create scenarios and then study people’s behavioral habits in a specific scenario.The current research lacks a general analysis method for user behavior in VR scenarios.This article categorizes user behavior in VR scenarios.Compared with general classification tasks,the input sources in virtual reality are more complex and diverse,and traditional time series classification models are difficult to handle this complexity.Due to the ability to obtain two types of data from VR scenes,one is user operation data and the other is scene images.This article designs two methods for VR user behavior classification based on these two types of data.A method of classifying VR user behavior based on operational data as a temporal data classification problem.A scenario based approach that considers VR user behavior classification as a video action recognition problem.This article designs a series of methods for VR user behavior classification and conducts experiments.The specific work is as follows:(1)For VR operation data,a temporal data classification method is adopted.A VR multi-source data user behavior classification model based on LSTM and attention mechanism was designed based on the characteristics of input data from multiple devices and different data structures in VR scenarios.This model obtains the overall information of multiple input sources through a summary method.Utilize attention mechanisms to integrate overall information and multiple input sources,including left and right hand controllers,headphones,and buttons.By using attention mechanisms to focus on key features,more accurate information can be provided and overall performance can be improved.This method achieved extremely high accuracy in VR operation datasets,with an accuracy improvement of 0.33% compared to the method using self attention.(2)Apply video motion recognition methods to VR motion recognition based on VR scene data.Since both video and scene can obtain a series of image data,this article uses video action recognition methods to process VR scene data,including 3D convolutional neural networks and timesformers.Since the VR scene is a first person perspective,there is no person in the scene picture.To obtain complete behavioral information,it is necessary to combine scene images and operational data.In response to this situation,this article attempts to combine scene images with operational information and design some fusion methods.According to the timing of fusion,it can be divided into early fusion and late fusion.In the later stage of fusion,two models are used to process scene images and operational data,and then the outputs of the two are fused.The early stage fusion is carried out before the final output,and the approach adopted in this article is to fuse the input data.This article designs a method for two-dimensional temporal data,which facilitates the fusion of the two.This article selects two fusion methods: additive and maximum.This fusion method improves the accuracy by 3.62% compared to 3D convolutional neural networks.This article obtains depth information while obtaining RGB images,and designs some methods for integrating RGB images and depth information.