Design Of Embedded System For Real-time Panoramic Image Processing

Only a limited field of view(FOV)is captured and showed by traditional photography and display technology.Meanwhile,360° FOV in both horizontal and vertical direction can be covered using panoramic technology.Users are able to choose specific angle of view to watch based on their needs.With the development of VR and AR technology,the combination of panoramic display and head-mounted display(HMD)has been widely applied in the areas of entertainment,tourism,education,industry and so on,which helps users get all-around viewing experiences.In military domains,fighter pilots or tank drivers are able to acquire information outside the closed space in real time using this kind of technology,which holds great strategic importance in the constantly changing battlefield.An embedded system is designed and described in this paper which processes the dual fisheye live stream captured by a panoramic camera to produce a window of perspective view from any direction in real time.Xilinx Zynq-7000 SoC is chosen as the processing core as it implements the framework of ARM+FPGA,which utilizes the advantages of sequential logic control of ARM processor as well as high-speed parallel execution and flexible configuration of FPGA.To solve the distortion problem of images captured by fisheye cameras,several projection models are discussed and the mapping between pixels in perspective and fisheye images is established.In addition,several image quality enhancement algorithms are discussed and implemented for antialiasing purposes.After the implementation of the aforementioned system,experiments are conducted to test its function and performance.Experimental results demonstrate that the system is able to produce undistorted perspective view window from any direction.A frame rate up to 25 per second can be achieved when the size of the window is 100×100,which meets the requirement of real-time display.