Research Of Projection-Based Interactive Storyboard With Sketch Cards

Traditional human-computer interaction is mainly based on keyboard,mouse and other devices for input and operation.With the development of artificial intelligence,augmented reality and other technologies,users prefer to interact with computers through multiple sensory channels to get a more natural and harmonious human-computer interaction experience.On the one hand,projection interaction breaks the spatial limitation of visual information presentation,and on the other hand,it provides users with a sense of touch when interacting with virtual and real objects.This thesis presents a projection-based storyboard interaction method with sketch cards,which enables users to interact with virtual objects by manipulating sketch cards.This interaction mode not only has the sense of control of physical interaction,but also greatly enriches the representation of the story scene through virtual elements.The specific work of this thesis includes:1.A multi-card tracking method combining Sketch-Alex Net and Kalman filter to achieve real-time tracking of sketch cards is proposed.The recognition network Sketch-Alex Net proposed in this thesis improves the original Alex Net network.It selects a large first layer convolution kernel to enhance the feature extraction ability,adopts a smaller step size to reduce information loss,and designs a set of convolution kernels with increasing factor to replace the original convolution structure,so that the network extracts features from shallow to deep levels.Training on the self-built freehand sketch data set shows good results.2.A card-based projection interaction paradigm and a dynamic storyboard projection interaction method are proposed.The paradigm is defined from four aspects: presentation mode,interaction object,interaction space and interaction behavior.The storyboard framework is designed and constructed from four aspects of role assignment,associated attribute,role control and role interaction,which supports the automatic status update of different roles and realizes the interaction between roles.By using the multi-card tracking results,the virtual character is projected onto the surface of the card,and the user develops the story by manipulating the card,and selects different cards to create their own story line to achieve dynamic narrative.3.User research and interaction evaluation are carried out for the system.From the perspective of user experience of human-computer interaction,an interaction evaluation system covering five indicators is proposed,including learnability,acceptance,interest,recommendation value and comfort.The participants were invited to experience the interactive method and accept questionnaire survey and structured interview.The results validate the effectiveness of the proposed card-based projection interaction paradigm and show that this article has a better learning experience,which can encourage children to be more actively involved in storytelling.