| With the vigorous development of display technology in the past few decades,the display screen has changed not only from monotonous black and white to colorful,but the display media has also developed in a variety of directions.In recent years,new technologies such as virtual reality and head-mounted display devices such as VR glasses have emerged,which makes panoramic images as their video sources popular in research.Innovation in display technology provides new possibilities for new types of panoramic displays.Therefore,in order to match the research of the new type of panoramic display in the future,a prototype of "spatial splicing panoramic display system" was proposed and designed in this paper,which extracts the panoramic image from the storage format with distortion and maps it to each splicer with spatial angle on the screen,so that a 360-degree panoramic image can be "projected" onto the "curved surface" formed by the splicing screen,giving the viewer a non-head-mounted "naked-eye" panoramic viewing experience.In addition,for the characteristics of panoramic display,deep learning super-resolution algorithm is applied to the system,which improves the resolution of the image and enhances the quality of the displayed picture.The main work and results of this topic are summarized as following:1)Hardware Design:Under the concept of the panoramic display system prototype,a demonstration system composed of 4display screens was built,and the full HD video segmentation design based on FPGA was completed.Based on DDR memory and FPGA,the video segmentation function of 1-channel 1080 P video input and 4-channel1080 P output is realized,which provides hardware support for spliced panoramic display.The throughput of5-channel 1080 P video requires high-speed calculation and high bandwidth occupation.In order to reduce the amount of data on the calculation,this design,to begin with,uses the timing symmetry of video storage and reading to combine the input and output buffers as one;secondly,the multilevel line buffer of the FIFO is used to achieve "Re-sampling Interpolation” in the circuit level.It allows the output resolution to be restored from half of the input to 1080 P to adapt to high-definition display devices.The entire function is designed by Verilog.The computing resources used are very little.The LUT occupancy is only 6 percent(11857/203800),and the LUTRAM occupancy is only 3 percent(1785/64000).2)Software Design:A)Design of spliced panoramic display algorithm based on video extraction.In this paper,the imaging model and projection characteristics of the panoramic image are studied.Based on the characteristics of multi-screen stitching,the "back projection" algorithm is researched and designed.It corrects the source image extracted from ERP format panoramic video through the algorithm,and allocates the processed image to multiple sub-screens as well,so that the display screen composed of multiple spatial splicing screens can meet the "perspective" impression.This algorithm can achieve any number of screen stitching.This paper uses 4 typical angles of screen stitching to verify its performance.In the test results,the average linearity of the picture reached 0.961,which basically met the viewer's "perspective" needs.B)Design of panoramic super-resolution algorithm based on deep learning.The limitation of the storage format of the panoramic image projection causes uneven spatial resolution.In this paper,in order to improve the appearance of the panoramic display,the application of the super-resolution algorithm in the panoramic field is studied.Based on the ESPCN algorithm,considering the image degradation model as well as the characteristics of the panoramic image,improvements of the algorithm has been made specifically in this paper: a panoramic data-set with panoramic degradation characteristics is constructed,and meanwhile the loss function is optimized,as well as the panoramic image quality evaluation parameters PW-PSNR are proposed.The PSNR value of the improved algorithm on the Panorama Pair dataset is 27.89 d B,which is1.03 d B higher than the SRCNN value of 26.86 and 0.92 d B higher than ESPCN.The PW-PSNR of the improved algorithm is 29.94 d B,which is 4d B higher than that of SRCNN and ESPCN.The entire panoramic display demonstration system has achieved good real-time performance.When the input panoramic source is 1080 P and only the display algorithm is used,it can achieve a frame rate of75 FPS,and the time consumption per frame is only 13.4ms;the super-score algorithm in the system also has real-time performance.When the input resolution is 960x540 and the output is twice as high as 1080 P,each frame takes 10.9ms and the algorithm speed is 17 times that of SRCNN. |
PDF Full Text Request