The System Of Video And Audio Transmission Based On GStreamer And Application On Robot

With the continuous popularity of smart devices and the wide coverage of wireless networks,there are more and more video service application scenarios.Audio and video transmission is widely used in many fields with its intuitive and convenient presentation and rich and exciting information content.Typical audio and video Transmission applications include smart homes,video surveillance,remote education,and telemedicine.With the continuous expansion of demand,especially with the rapid development of mobile communication technology,research on real-time video transmission systems that can be efficiently and flexibly deployed in the mobile communication environment is more urgent.There has been little research on the synchronous transmission of audio and video in the GStreamer framework.This paper proposes an audio and video wireless transmission system based on the GStreamer framework,designs and implements one-way audio and video synchronous transmission and two-way voice interaction,and studies the application of the system in mobile robot scenarios.The system uses the Raspberry Pi 3B as the hardware platform,uses the V4L2 driver framework to collect video data,and uses H.264 video encoding;the audio data is captured by the Pulse Audio sound server,and the Opus audio encoding is used.The audio and video data are packaged according to the RTP protocol and completed through the UDP protocol data transmission.The test results show: System has high real-time performance,in WIFI audio and video synchronization transmission delay is261 ms,in 4G audio and video synchronization transmission delay is 330 ms,and only video transmission is 136 ms and 206 ms,respectively.The transmission quality test uses a non-reference image spatial quality evaluator and a MOS scoring standard,system image quality evaluation is 40% higher than the comparison scheme.According to different application requirements and network environment,the system realizes multi-resolution optional video acquisition and automatic switching transmission between Wi Fi and 4G network,which improves the flexibility and economy of use,meets the needs of many scenarios,and has good reference value for audio and video transmission applications.First,the analysis and research of key technologies is conducted.Introduced GStreamer open source multimedia framework,analyzed and studied its advantages in audio and video transmission.And analyze and research the key technologies of audio and video transmission,including video compression coding technology,audio compression coding technology,RTP real-time transmission protocol,TCP and UDP transmission protocol.Next,the system design and application research are carried out.The system design is divided into collection end design and client end design.The design goal is to achieve one-way video transmission and two-way voice interaction.The collection end of the system is based on the Raspberry Pi platform.The software design includes two pipelines,audio and video transmission pipeline and audio reception pipeline,and the automatic switching function of 4G network and WIFI network is designed for the collection terminal.System client software design includes audio and video receiving pipeline and audio transmission pipeline.The research and realization of the application of the system in the mobile robot scene are carried out.Finally,the system is tested and analyzed.Contrast test of the system's real-time transmission and transmission quality,analyze the advantages and disadvantages of the system,and verify the application of the system on mobile robots.