Research On Synthetic Aperture Imaging Method Based On Semantic Inpainting And Super-resolution Reconstruction

Synthetic Aperture Imaging is an important branch in the field of light field computational photography.It overcomes the limitations of the traditional singlecamera imaging,and then achieves the effect of "seeing through" occlusions,and it is one of the hotspots of computer vision field research.In application scenarios,Synthetic Aperture Imaging captures original visual information through various methods such as vehicle movement and airborne aerial photography.It can be widely used in areas including object detection and security monitoring.With the improvement of the camera array system,Synthetic Aperture Imaging has also been continuously developed.However,improving the imaging quality of occluded objects has always been a concern of domestic and foreign researchers.First,in the process of imaging using the original information captured by the camera array,if there is a situation where the visual information of the occluded object region is not captured by any sub-camera view in the camera array,it will result in a lack of information in the focused imaging region,and it greatly affects the Synthetic Aperture Imaging quality.Secondly,when the original visual information captured by the array camera is in a low-resolution scene,the resolution of the synthetic aperture image will be lower,which further reduces the quality and accuracy of thesynthetic aperture image.(1)Semantic inpainting of missing information regions in background estimation.Existing background estimation algorithms will fail when the number of viewing angle images is small or part of the background is heavy occluder.Therefore,determining the failure region in the estimation result and inpainting on the missing information region to generate a clear and accurate background is a necessary work to improve the quality of Synthetic Aperture Imaging.(2)Semantic inpainting of the occluded object in the focus plane.None of the existing Synthetic Aperture Imaging methods can handle the situation where the visual information of the scene is not captured in all the camera array angles of view,which will result in the lack of information when the object is focused on the plane where the occluded object is located.Therefore,The missing information region is further completed on the occluded object based on the semantic inpainting method,which effectively guarantees the global and local consistency of the inpainted image,thus,generating complete and accurate results to improve the object imaging quality.(3)Synthetic Aperture Imaging blur problem with low resolution input images.Because the camery array usually contain dozens or even hundreds of sub-camera,thus it high cost of some capture scenes.When the low resolution image is used as the source input under this condition,the object region in the synthetic aperture image is not clear due to the lack of relevant texture information in the scene.Therefore,a deep reconstruction network based on convolutional Long Short-Term Memory network is used to super-resolution reconstruction of low-resolution images by using image association information from multiple perspectives to obtain high-resolution images.On this basis,the relatively clear occluded object can be obtained by refocusing to the plane of the occluded object,and the imaging quality of the object can be further improved.The experiments in the public dataset and our dataset,it is shown that the research method proposed in this paper can effectively improve the imaging quality of the occluded object in the synthetic aperture image.At the same time,qualitatively and quantitatively results in our method are compared with the related work results of researchers in this field.The results show that the research method in this paper can effectively improve the quality of Synthetic Aperture Imaging.