Research On Video Voice Recovery Based On Micro-vibration Extraction Algorithm

Vibration exists everywhere,and it produces the emergence of the concept of sound.The sound generated by vibration can not only be heard directly but also can be received through a microphone and converted into an electrical signal.In recent years,new sound recording technology has attracted widespread attention.Its basic principle is the same as that of the traditional microphone.It records sound by receiving vibrations,but the difference is that this technology records sound through video,which is called a "visual microphone".The principle of "visual microphone" is to record the micro-vibration caused by the sound wave mapped to the object through the camera and other equipment,and then extract the micro-vibration signal recorded in the video through the corresponding microvibration extraction algorithm,and then process the signal enhancement to restore the sound signal.The micro-vibrations caused by sound waves are often pretty weak,so the key to achieving "visual microphones" is vision measurement technology and microvibration extraction algorithms.Therefore,one of the problems to be solved in this thesis is how to extract the tiny vibration signal more accurately from the published high-speed video and enhance the Micro-vibration signal,and finally restore the voice information;the other is how to realize the micro-vibration signal through the low-speed video shot by the ordinary camera.The vibration captures and restores the low-frequency signal in it.This thesis will use two different methods to extract the micro-vibration of high-speed video and low-speed video,so as to restore the sound signal.The research content mainly includes the following two aspects:1.First of all,this thesis studies and implements the method of voice recovery based on high-speed video.The "visual microphone" is improved and optimized,and a video voice recovery method based on single-channel RGB components is proposed.The PVMbased micro-vibration extraction algorithm is used to realize the voice recovery of the RGB single-channel component.The relevant experiments are carried out using the highvoice video data set publicly available on the Internet,and the experiment proves that the video voice recovery effect based on the single-channel RGB component is better.Objectively evaluate PESQ scores and higher signal-to-noise ratio scores.2.Secondly,this thesis studies and implements the sound recovery method based on low-speed video.A video sound recovery method based on phase consistency edge detection is proposed.The vibration extraction of low-speed video is realized by using the micro-vibration extraction algorithm based on edge detection.The principle is that it uses an ordinary SLR camera to take video firstly,then,selects the target area by video preprocessing,and extracts the target edge contour by edge detection algorithm to obtain the amplitude signal of target edge vibration.Finally,it restores the sound by enhancing the signal.The experimental design and experimental results show that the method proposed in this thesis can successfully extract micro-vibration signals and restore lowfrequency sounds.