Research On System Constructing And Roaming Technology In Mixed Reality Apps In Large Indoor Spaces

In recent years,mixed reality technology is becoming more and more mature and widely used in games,medical treatment,education and other fields.With the development of hardware and software technology,more and more mixed reality devices emerge,such as Microsoft HoloLens and Magic Leap One.Mixed reality blends real world and virtual world.To create more realistic user experience,the device need accomplish scene reconstruction and scene registration.Existing devices such as HoloLens provide users with spatial mapping module.Users can complete a scan of all parts of the environment with HoloLens and HoloLens will reconstruct virtual scenes using multi-view geometry reconstruction.For mixed reality applications used in a small scope,the process of scanning can provide accurate environmental mapping.However,for large indoor spaces,the scanning process has many limitations,including limited scan scope,consuming time and effort.In addition,the computing resources of existing mobile mixed reality devices are limited.So,for large indoor spaces,with the enrichment of virtual scenes and contents,the system performance will decline in the process of users'real-time roaming,then rendering delay will occur.The existing mixed reality applications are mostly limited to a small scale of interior space,and a good solution to above problems in large indoor spaces such as museums and art galleries has not been proposed.In addition,the virtual scenario models in most mixed reality applications are designed by professionals using professional tools and then deployed to devices at present.These operations require the technical staffs that have professional basic knowledge and practical experience to complete,which is difficult for ordinary users.Therefore,this paper proposes a method to fast and easily construct mixed reality applications,especially for large indoor spaces with complex structures.At the same time,it ensures that users have smooth real-time roam experience.It effectively solves the problem that existing mixed reality devices,such as the HoloLens,are unable to scan and model an entire large scene with several rooms at once.This paper also proposes a data structure called VorPa,based on Voronoi diagram and four roaming strategies,including path generation and edit strategy,layout optimization strategy,rendering acceleration strategy and collision detection acceleration strategy.These strategies effectively solve rendering latency in the process of real-time roaming in large-scale spaces and improve the user's experience of reality and immersion.The main contributions of this paper are summarized in the three following parts:1.This paper provides a constructing method facing MR apps in large indoor spaces.Reconstruct virtual scenes using building information which is easily to get and set accurate mapping relationships between real and virtual spaces.This method solves the problem that present mixed reality devices cannot rapidly model and register large indoor spaces with several rooms through scanning.2.This paper proposes a rendering acceleration strategy for MR apps in large indoor spaces.It includes a data structure VorPa based on Voronoi diagram and a rendering acceleration strategy based on VorPa.This strategy uses VorPa data and user location as constraint conditions,achieving replacement of models with different qualities and ensuring that the complexity of scene doesn't exceed the pre-set threshold.In addition,this paper proposes path generation and edit strategy,scene layout optimization strategy and collision detection acceleration strategy to further optimize rendering speed,ensuring real-time roaming experience.3.This paper proposes spatial anchor deployment strategies supporting deviation correction between real and virtual spaces.Spatial anchors can maintain virtual object's location and orientation in real world.Using this property,we set several spatial anchors in scenes and bind virtual models to nearest anchors to correct deviation.To reduce the power consumption of system and ensure the experience effect,this paper designs two spatial anchor deployment algorithms to arrange the layout of spatial anchors and satisfies the conditions including minimum number of spatial anchors and constraints of the use of spatial anchors,which respectively are spatial anchor deployment algorithm based on greedy algorithm and spatial anchor deployment algorithm based on Integer Linear Programming.It is proved that the spatial anchor deployment algorithm based on Integer Linear Programming has better effect in the aspect of fewer anchor amount and shorter running time.