Study On Sound Synthesis Of Outdoor Water Scenes

With the continuous development of computer graphics technology,people have paid more attention to the sound quality of video and animation.Besides the work of dubbing for characters,we call the generation of other sound effects as Foley.Most of the work is done manually by experienced Foley artists.In order to automatically complete the Foley work,taking the water scenes as an example,the existing work mostly uses physical-based methods to generate sound for small-scale water bodies.This method has high computational complexity and is not suitable for outdoor water scenes.To this end,this paper proposes two sound generation methods which can be applied to outdoor water scenes by using non-physical sound generation technology and deep learning technology,respectively.Existing physical-based sound synthesis methods need to simulate and extract bubbles in water(the main source of sound in water scenes),which has high complexity.In order to improve the efficiency of the algorithm and enable it to be applied to sound generation in outdoor water scenes,an example-based sound generation method is proposed in this paper.This method is used to generate sound for ocean waves.The classification mechanism based on particles' depth is used to replace the more complex bubble simulation method in existing algorithms,and the bubble particles in ocean waves are generated efficiently.Then,a large number of bubble particles are extracted and partitioned into a few grids to reduce the computational complexity in the process of wave clustering.By mapping the attributes of ocean waves clustered by the greedy algorithm and parameters of sound clips,this paper finally realizes the efficient and high-quality generation of ocean waves sound by using the given sound examples provided by users.This method is based on the characteristics of the wave scenes.When the liquid simulation method or simulation object changes,this method can not be directly applied.In addition,for the water scene video which can not obtain the liquid state,its corresponding sound can not be generated by the above method.In order to avoid the limitations of the above method,this paper designs a water scene sound generation method based on deep learning.This method can automatically learn the relations between different states of water scene and sound in video,thus realizing an end-to-end generation from a water scene video to its corresponding audio.Because of the difficulty of this cross-modal task,this paper chooses the GAN model with strong learning ability as the prototype of the whole network.After the video preprocessing,we train the generator and discriminator designed in this paper.Finally,this method can output sound corresponding to the input water scene video.In order to further improve the audio quality,we design a timbre enhancement network to optimize the final sound results.In addition,we put forward two evaluation indicators for quantitative evaluation of the quality of the generated results.The experimental results show that the algorithm can generate the sound of many kinds of outdoor water scenes,including waterfalls,waves,rain,and so on.The results are realistic.