The Design Study Of Natural Interactive Gesture Based On Image-schema Coding

The advent of artificial intelligence requires computers to be more sensitive to the ambient environments and the user intentions,thus the conceptual boundaries of the interaction between human and object has been greatly blurred.This trend also enriched the content and means of human-computer interaction(HCI),and redefined the design research of natural user interface to some degree.Regarding the input modes,the pervasion of sensing devices and the ubiquity of interaction scenarios enable people to express the personal preferences for interactive inputs,especially gestures,to ensure "natural" experience of interaction.Thus the thinking models of user in terms of transferring prior knowledge to the new interface will be more diverse.The basic research question in this paper is: how to extract the common features of conceptualization from cognitive processing of users and transform them into design elements,so that designers can gain insights into the relatively natural form-meaning relationships of gestures in HCI for users and then develop the gesture sets which not only reflect those relationships but also have good usability and access to technical supports.This big question includes two research subtopics:(i)What are the common features of cognitively mapping gesture performance to its corresponding interactive intention by conceptualizing.In this paper the image schemas are considered to be condensed descriptions of the common features by means of organizing,systemizing and abstracting human experience.(ii)How the image schemas are used in defining interactive gestures and how the image schemas play a role in categorizing user-defined gestures.The research contents of this paper are as follows:Chapter one defines the research questions,objects and key terms,and confirms the research fields which include HCI,cognitive psychology,cognitive linguistics,user study,etc..By clarifying relevant concepts and summarizing the existing studies,the main idea and technical roadmap are well specified.In Chapter two,a theory of using image schemas as the general character of understanding interactive gestures is presented.On this basis,the protocol analysis of verbal descriptions of user gestures can be an approach to identify the use of underlying image schemas.Through the literature review and case study method,an important relationship between the characteristics of image schemas and the interpretation of text.Chapter three introduces the concept,procedures,rules and essentials of an working method of coding gestures with image schemas.The procedures,rules and essentials of this method which guides designers to extract image schemas also lead to a design method about categorizing user-defined gestures before evaluating the representative gestures.Case studies and empirical researches are conducted i n order to justify the feasibility and meanings of the design method,and to explain its execution details.In Chapter four,the design method is applied in gestural interaction scenes to select suitable user gestures.Afterwards,the performances of these gestures are measured by simulating real environments and designing experiments,with the aim of verifying the design method through comparing the outcomes in different conditions such as user interface and mental state of participants.Chapter five focus es on the effects of scenario factors on the activation and use of image schemas.The scenario factors have impacts on the dynamics of gestures that express the image schemas which connect different cognition domains.This study suggests that the combination of user elicitation and scenario-based design framework optimizes the identification of the transfer mode of user knowledge with more details of interview,thus improving the accuracy of stated preference and image-schema coding.The results and innovations of this study can be summarized into three points as follows:Firstly,combing with the theoretical model of double-layer information nodes,the study describes the mapping relationship of four cognition domains which exist in the thinking model of defining the form-meaning relation of a gesture,based on the famous conceptual metaphor theory.The consistent mapping from semantics of a gesture to its movement is maintained because these cognition domains share a limited number of image schemas.On the other hand,image schemas relate the gestural semantics to the meaning of interactive tasks.Secondly,this study proposes four strategies of extracting image schemas from transcripts according to the relevant theories and literature of cognitive linguistics,as the value of identifying image schemas of user gestures is discussed above.To find the guidance of interpreting gestures with image schemas,this paper argues that some linguistic elements(e.g.,verbs and prepositions)which involve dynamic and spatial semantics and some use of metaphors are the access to judge image schemas.By tagging a gesture with typical image schemas,we can know the types of gestures in case that they are categorized by image schemas which are used to construct the form-meaning relation of gestures.These image schemas also represent the thinking models that users define,understand and recall gestures.The purpose of selecting a representative gesture in each type is to motivate users to evaluate the experiences of applying different thinking models.In this fashion,participants indirectly rate the various cognitive structures according to their own habits and preferences,whilst design experts are in charge of selecting gestures to ensu re the usability of representative gestures.Thirdly,this paper reports the designing of gestures for tangible user interface of a desktop intelligent device and for the hands which are grasping objects,after analyzing the design space of interactive gesture.The results show not only positive effects but also shortcomings of the design method.Further researches consider that the use of image schemas contributes to analysis of user's cognition from the perspective of design methodology.