Research On Personalized Puppetry Generation Based On Depth Sensibility

Chinese shadow play is an important Intangible Cultural Heritage of world-class, it has a significant value on its aesthetics, protection and research. In this study, we take “Jiangbei School” Wangkui shadow play as the representative of Chinese shadow play, and propose methods of generating personalized puppet figure and somatosensory interacting with virtual puppet based on depth sensibility.Personalized puppet figure integrates facial features of the user into puppet facebook, which stylizes human face of shadow play. In order to describe the characteristics of users’ personalized face, this study combined AAM algorithm with depth image to obtain facial feature points in real-time, and establish vectors and parameters as the morphological description of profile contours and facial features. After obtaining three-dimensional facial features, scale transformation of 2D face coordinate system and image coordinate system, generation of puppet style feature points, curve fitting, and image transformation based on feature parameters, we ultimately generate personalized puppet figure.We apply personalized puppet figure to the somatosensory interaction module, in which users’ skeleton data based on depth sensibility is utilized to drive the motions of virtual puppets. To preserve traditional motion characteristics, we analyze the structure and motion of Wangkui puppets, and propose two somatosensory interaction methods, which are stylized motion mapping based on motion recognition and fallow motion mapping based on joints coordinate.Finally, the paper interprets the design and implementation method o f the system, and study the system performance and cultural protection function in aspects of the similarity of puppet figure, the usability of somatosensory interaction, and the effect of cultural transmission.