Interaction Design And Application Study With Mixed Reality Glasses

In recent years, more and more Mixed Reality(MR) devices were developed, no matter hand-held devices or wearable devices. While when it comes to the interactive styles, user interfaces and applications of these devices, the researches are still ongoing. The user interfaces of MR devices are still 2D for most. Although it’s clear and simple, it doesn’t match the 3D space where users are; In addition, there are plenty of researches on interaction styles of these devices. Some of them interact through body,audio, tactile sensation and other human sense, while others are based on Augmented Reality(AR) agent like pen with pattern. The methods are all different, and what matters is whether it’s suitable to application and the user experience. Apart from that,many AR and MR application has the function of importing models. It’s convenient to import models, but it’s not personalize. There are many researches on creating models in AR and MR application either. The main di?culties are how to avoid the shortness of people’s modeling ability and how to maximize people’s freedom to create. This work focused on these problem:(1)the fabrication of a MR glasses,(2)the 3D user interface,(3)the user interaction style based on gestures, and(4)3D modeling with bare hands, which is widely used in applications. Evaluation are presented after design.The followings are main work in this paper:(1)First, a video see-through MR glasses is made by ourselves, which is used for the rest of our work.(2)Palette-like 3D menus and three kinds of placement are designed and implemented based on MR glasses to meet different ways of thinking, such as palm-based menus, object-tracked menus and screen-?xed menus.(3)3D basic manipulation method mainly based on gestures are designed and implemented, and head motion is supported for assistance. In addition, to improve the user experience, different assistance methods including head constraints, skeleton balls and board are added to support user’s manipulation.(4)At last, a new method of creating 3D models, which is familiar in all kinds of AR and MR applications is presented. We created our own principles for bimanual interaction on 3D modelling based on the hands-free devices and former principles and the 3D modeling application with bare hands. We pinpoint the shortage of previous work and improve people’s modelling result even they are bad model creators by a seamless modelling environment among different status switch, and enough freedom for people to create their own models.After all, we evaluate the feasibility and usability of every part of the work by experiments, which includes user interface, manipulation and 3D modelling. We could safely draw the conclusion on user’s feedback and data collected in experiments that the palette-like menus and interactive styles for manipulation are suitable under MR glasses, and the application of 3D modelling by two bare hands has got nice feedback as well.