The Design And Applications Of Natural Interaction Based On Event Prototype Derivation

From Command Line Interface (CLI) and Graphical User Interface (GUI) totoday’s Natural User Interface (NUI), the mode of human-computer interactions hasexperienced a tremendous revolution. Technology companies such as Apple andMicrosoft seized the historical opportunities to develop their natural interactionproducts like Kinect, Wii, and Siri, which forcefully promoted the furtherdevelopment of interaction design. While receiving the huge commercial interests,the Tech companies also offered new possibilities for interaction design in the future.Bill Gates, founder of Microsoft, once boldly predicted that mouse and keyboardwould be gradually replaced by the more natural ways of interaction, as he foreseenits broad development prospect. At present, the research and practice on naturalinteraction design from the academic field of Designing Art are still in the start-upstage and there is an urgent need to build a systematic theory framework that hasguiding and referential significances on both analysis and practice of design.The topic of this article originates from the two basic questions faced by research,teaching and practice of natural interaction design: First, what’s the basic goal ofnatural interaction design? Second, what might be the systematic and practicabledesigning method for realizing the goal?According to the inherent requirements of natural interaction design, this articleinvestigates the relationship of information decoding and coding among designer，interface and user from the perspective of information transmission. Based on thedual evidences of literature researches and case studies，and through related theoriessuch as cognitive linguistics, cognitive psychology, and semeiology, the articleexplores the theoretical foundation of making the “Event Prototype” of user’spreexistent knowledge carrier as the switch media of natural interaction design and itsderivative external form and internal mechanism. Via the process of cross-domainmapping of cognitive elements (including characteristic information node andsemantic information node) in the event prototype derivation，the article discovershow to design "a meaningful natural interaction event" for users，namely, to establisha complete cognitive motivation chain between user’s pre-existing event prototypeand objective interaction events.It also constructs the natural interaction designing methods from the point view of information coding of visual channel on the basis of the derivation theory of eventprototype in order to meet the research，teaching and practice needs of Designing Art,and it verifies the feasibilities of this designing method through gesture interactionpractices and experiments in Virtual Reality environments.The main innovations in theories and practices from this article are as follows:Firstly, it brings up the concept of Event Prototype in natural interaction designand the analysis approaches of Event Prototype Derivation.This article systematically analyzed the hierarchies and components of cognitiveelements of event prototype. It proposed that the event prototype of naturalinteraction mainly originates from the real interaction event and the pre-existingdigital interactive event. Prototype is the unification of the “form layer” of externalcharacteristic representation and the “meaning layer” of internal pre-existingknowledge composition, therefore, the cross-domain mapping between theinformation node on the “form layer” and the information node on the “meaninglayer” becomes the basic analysis technique of event prototype. Through the cognitiveprocedure which includes "prototype recognition"，"prototype matching" and“prototype inspiration" of the target domain, the cognitive subject activates hispsychological heritages by cognitive elements which come from the “form layer”,thus, realize the targets of natural interaction and multiplexing of pre-existingknowledge.Secondly, this article systematically analyzed the natural interaction cases byutilizing the event prototype derivation analysis and summarized the basic mode ofevent prototype derivation.Through the analysis on the diachronic and synchronic of natural interactioncases, this article discovers that the derivation approach of interactive event prototypeis the cross-domain mapping of cognitive elements and the unification of theinheritances and developments of event prototype. The internal and externalmotivations of prototype inheritance and development originate from the developmentof demands in the interactive context and the changes of interactive modes,represented by the evolution form of micro evolution and macro evolution in theprocess of prototype derivation and the variants of different levels of familyresemblance. The deconstruction and reconstruction of cognitive elements in theinteractive event prototype is the basic pattern of event prototype derivation. Themetaphor and metonymy based on image schema provide cognitive strategy for eventprototype derivation and also form the cognitive motivation chain between “Event Prototype" and "Derivative Variant".Lastly, with the double evidences of literature and case study, this articleproposed the natural interaction designing method based on event prototypederivation and applied it in practices and experiment verifications. This relativelycomplete designing method includes event prototype acquisition, prototype derivation,concept generation of natural interactive event, coding and iteration. Based on theevent prototype derivation variant created by the Cross domain mapping of eventprototype from the double layers of information nodes, in other words, naturalinteraction event, designers design event prototype derivatively according to thesituation of target domain. By utilizing this user-centered method, designers build acomplete cognitive motivation chain between event prototype in source domain andderivative variant event in target domain in order to design "the meaningful naturalinteraction events" for users， thus, to realize the cross-domain reuse of user’spre-existing knowledge and the target of natural interaction.This article imported the concept of “Event Prototype” to natural interactionanalysis and design and enabled designers to follow and apply the cognitive rulesconsciously in order to help users reusing their pre-existing knowledge in thecross-domain cognition. Optimistically, this article would contribute the innovativeand feasible ways of research and design to the vigorously developed naturalinteractions.