Research And Prototype Design Of Smart Beaker For Fusion Experiment Teaching Of The Fusion Of Virtuality And Reality

The virtual experimental system is based on virtual reality and other technologies to build an experimental environment that combines the real world and the virtual world.Experimenters can complete various experimental projects by interacting with experimental equipment.With the continuous development of our country’s higher education,experimental teaching has become an important teaching content stipulated by the national curriculum plan and curriculum standards,and the virtual experimental system has become an important teaching tool for middle schools.The virtual experiment system can not only solve the problems of high experiment cost and dangerous experiment operation in middle school chemistry experiments,but also help students better understand experiment principles and memorize experiment procedures.However,the interaction mode of the existing virtual chemical experiment system is relatively single.This single-modal interaction method is not only prone to misidentification problems caused by irregular operating behaviors,but also easily increases the user’s operating load,and cannot give users a true experimental experience.Therefore,under the support of the national key R&D project "multimodal natural interactive virtual and real fusion open experimental teaching environment",this paper takes "theoretical teaching and practical teaching complement each other,and the physical experiment and virtual simulation are combined" as the design goal and takes "multimodal fusion and intention understanding" as the design orientation.In order to break the limitations of traditional experimental teaching,allow students to conduct experiments at any time,and increase the interaction and immersion of experimental operations,this paper designs a new type of smart beaker structure with perceptual ability,proposes a multimodal fusion and intention understanding algorithm based on the fusion of decision-making layers,and realizes the smart beaker chemistry experiment system.The contributions made by this paper mainly include the following three points:(1)A smart beaker model for virtual chemistry experiment teaching is proposed.In order to improve the accuracy of user behavior perception in the virtual chemical experiment system and the authenticity and immersion of user experimental operations,this paper designs a set of smart beaker structure with perceptual ability,and realizes its functions through 3D printing technology and sensor embedding technology.The smart beaker is equipped with a variety of sensors that can detect user behavior,including: photosensitive sensors,sound sensors,posture sensors,binocular cameras,etc.,so that it has functions such as object perception,voice perception,scene perception,and situational awareness of the beaker itself.It solves the system’s detection of the dumping,movement,reagent selection and other functions of the beaker during the experiment.The experimental results prove that the use of the smart beaker improves the user’s sense of experience and immersion.(2)A new multimodal fusion intention understanding algorithm and navigational interaction paradigm is proposed.Aiming at the misrecognition and unreality of a single modality,this paper proposes a multimodal fusion intention understanding algorithm and navigational interaction paradigm based on the decision-making layer.First,according to the results of multi-channel(visual,audio,and touch)intention perception,the three modal information are processed,and a multimodal information database is constructed to clarify the mapping relationship between different behavior input information and intentions.Then,the multimodal information is fused through the multimodal fusion algorithm based on voice perception and sensor perception and the multimodal fusion algorithm based on scene perception and posture perception.At the same time,a navigational interaction paradigm is proposed based on the multimodal fusion algorithm,and an intelligent navigation system is realized.Experimental results prove that the two multimodal fusion intention understanding algorithms proposed in this paper can accurately understand user behavior.The use of intelligent navigation system can improve the user’s learning ability of the smart beaker and reduce the difficulty of using the experimental system.(3)The smart beaker chemistry experiment system with error display ability and experiment exploration ability is realized.Based on the structure of the smart beaker,the multimodal fusion intention understanding algorithm and the navigational interaction paradigm,this paper completes the smart beaker chemistry experiment system and designs 8chemical experiments for verification.The innovations of our smart beaker chemistry experiment system are as follows: First,the system can provide corresponding scene information feedback according to the amount of different experimental reagents,allowing students to independently conduct exploratory experiments;Second,when the user’s behavior is not standardized or correct,the system can give corresponding operation prompts,and there will be feedback on the error experiment phenomenon;Third,the system adds a record of interesting questions and answers and screenshots of experimental phenomena,which can help students master knowledge points,and the record of screenshots of experimental phenomena can facilitate teachers to check the completion of the experiment.The experimental results prove that the smart beaker chemistry experiment system can not only meet the basic needs of traditional chemistry experiments,but also complete error cases that cannot be tried in traditional chemistry,which can be better used in experimental teaching.