Research And Development Of Tujia Virtual Instrument System Based On Gesture Interaction

Chinese culture has been extensive and profound for five thousand years,that Chinese instruments have a long history.Different nationalities have their own history of musical instruments.With the development of modernization,traditional instuments have taken a serious hit.As standards of living have risen,the spiritual word has demanded and more attention has paid to traditional instruments.Tujia instruments have been listed as world heritage sites.With the development of science and technology,it has becomed a new trend to inherit and develop traditional culture in a digital form.The use of digital forms to protect traditional culture is undergoing various explorations.There are computer virtual instruments based on Midi(Musical Instrument Digital Interface)technology and multi-modal human-computer interaction technology.The main research of this article mainly includes the following aspects:(1)In the research in this article,Tujia instruments with ethnic characteristics were selected.The research analyzes the status of virtual instrument research,puts forward the existing problems and finds effective solutions.This study has important reference and use value for the inheritance and protection of intangible cultural heritage.(2)The research content of this study is mainly development of Tujia virtual instrument System based on gesture interaction.Research on gesture interaction methods for virtual instruments.First,it is the study of gesture interaction based on Hidden Markov Models.It describes the theoretical knowledge and three basic problems of Hidden Markov Models.The Customization of Tujia Percussion Instruments' Gesture and the Extraction and Optimization of Dynamic Gesture Features,that affects Hidden Markov model initialization,training and recognition results.The second study is gesture interaction research based on deep neural network.it introduces the basic concepts of deep neural network and training deep neural network.Both methods will eventually analyze the process and results of the experiment and prepare for the later study.(3)The main researches of this study research on gesture interaction method for virtual instruments,the design of Tujia virtual instrument system,the realization and test of Tujia virtual instrument system.Research on gesture interaction method for virtual instruments uses Hidden Markov Model and deep neural network.The design of Tujia virtual instrument system was based on analysis of user needs and functional requirements.First of all,the three-dimensional model of the Tujia virtual instrument is constructed.And the traditional interactive method of virtual instruments and the interactive method of gesture interaction are studied.Finally,the system saves a MIDI file when playing the virtual musical instrument,and analyzes the MIDI file to play the virtual musical instrument automatically.Next,realization and testing of Tujia virtual instrument system.Implemented the entire virtual instrument system on the development platform of Unity3D,including the system interface design and interactive mode,automatic performance,music preservation and other functions.through functional testing,compatibility testing,interface testing to ensure the reliability of the virtual instrument system.(4)The study suggests effective solutions on the research of virtual instruments at home and abroad is that instrument model construction is simple,most of virtual instruments lack automatic performance research and a single interactive way,poor user experience.,we explored a service model by advanced methods for the protection and inheritance of Tujia instruments that can be used in practice.