Research On Real-time 3D Imaging And Display For Unmanned Aerial Vehicle

With the rapid development of UAV in recent years,the application of UAV real-time 3D imaging and display technology in military and civilian fields is rapidly expanding,it is widely used in surveillance reconnaissance,disaster relief,obstacle detection and other fields.The purpose of this paper is to build a universal and low-cost real-time 3D imaging and display system for UAV,the main work and contribution of this article are as follows:(1)In order to simulate the working principle of human eye 3D imaging,a binocular camera system with parallel optical axis is designed in this paper,the binocular camera is calibrated using calibration plate to obtain a binocular image with parallel optical axes.(2)The binocular image acquired by FPGA is Bayer format,in order to obtain more realistic color image,this paper takes full advantage of the FPGA's high parallelism,pipeline operation and so on,the bilinear interpolation algorithm is adopted to recover the Bayer format image to a color image,the color distortion of the color image is solved by the color correction algorithm,thus the color correction of the binocular image is realized.Aiming at the problem that the 3D imaging algorithm of SFM can only reconstruct the outline information of the scene and it is difficult to reflect the details of the scene,a 3D imaging method based on human vision is designed in this paper.The two channels of color image are downsampled by FPGA,then the two video images are spliced into a signal image,3D display is realized by 3D display device.This method employs less system resources,but has high real-time performance and good 3D imaging effect.(3)In order to transmit 3D images on the UAV to the ground for 3D display in real time.On the embedded ARM platform,a H.265 video encoder based on GStreamer framework is designed to realize real-time compression and transmission of 3D images.A H.265 video decoding and display software based on the FFmpeg+SDL framework is designed on the ground processing platform.The displayed video image is output to the virtual reality glasses through the HDMI interface for 3D display.The method achieves long-distance transmission and display of 3D images through a limited transmission bandwidth while ensuring a certain image quality.(4)System design and implementation.Because of the small size of the UAV platform and the limited power consumption,the embedded hardware platform of FPGA+ARM is designed on the UAV to achieve real-time acquisition of binocular images,color correction,3D imaging and H.265 video compression.At the same time,the H.265 video stream decoding display is realized on the ground CPU platform,the display of the image through the virtual reality glasses to realize the 3D display.The experimental results show that the system can achieve 3D imaging and display for video images with resolutions of 1920~*1080 and frame rate of 30 in any scenario.The system delay is about 300 ms,which meets the real-time requirement.