Research And Application On 3D Interaction Techniques Based On Semantics In Virtual Environment

Virtual reality(VR)is becoming a new computing means.It has wide potential applications in many important areas such as national defense,scientific research, engineering design and manufacturing.From the viewpoint of human-computer interaction,VR comes forth as a three-dimensional user interface(3D UI),which is very different from 2D WIMP interfaces and confronts with a lot of new challenges and more complex usability issues.Many existing 3D UIs hardly meet the actual application requirement because they are too simple or lack usability.Therefore,we must carry out more deeply research about high usable 3D interaction techniques (ITs).From the perspective of application,the lack of natural and harmonious interactive modes has become a "bottleneck" restriction for the further application of VR.Based on the combination of multimodal interaction and 3D interaction,the study on multimodal 3D ITs has become a research focus.In this dissertation,the existing 3D ITs are presented,analyzed,and summarized from the viewpoint of design-level.The support provided by the existing common task-oriented ITs to the application is insufficient.Therefore,in this thesis,based on the concepts of scene semantics and semantic objects,the definition of scene semantics is put forward.Some 3D ITs are designed based on scene semantics,which make complex application be carried out by the users' basic interaction with semantic objects.The validity of scene semantics is proved by a practical application-"virtual furniture layout".Using multimodal 3D interaction is an effective means to achieve natural and harmonious VR systems.Thus,interaction semantics is defined from the view of multi-channel input.A new hierarchical integration algorithm is proposed with reference to the existing theory of task syntax analysis so as to realize multimodal fusion.Some 3D ITs are designed based on interaction semantics,and the validity of interaction semantics is confirmed by a practical application-"virtual city planning".The three innovative ideas in this dissertation are summarized as follows. 1.Scene semantics is defined from the viewpoint of application.Based on the concepts of scene semantics and semantic object,complex application can be carried out by users' basic interaction with semantic objects.The designing policy of scene semantics is proposed,meanwhile some 3D ITs are designed and carried out based on scene semantics.2.Interaction semantics is defined from the viewpoint of multi-channel input. A corresponding integration algorithm is proposed to realize multimodal fusion.And a mechanism for the integration of interaction semantics is proposed based on time-constraints.3.Based on scene semantics and interaction semantics,natural and effective two-handed 3D ITs and multimodal 3D ITs such as seamless world-in-miniature,dynamic grouping,fast multi-object select and detail-skipped shape arrange are designed.