Research On Under-screen Imaging Of Smart Phone

The smartphone has become a daily essential,and its role in human life is becoming more and more important.Its functions and appearance are constantly improving.In recent years,"full screen" has become a research hotspot of mobile phones,and the front camera of mobile phones has become the last obstacle to a truly full screen.Since the screen of a mobile phone cannot be completely transparent,the circuit traces and other opaque parts in it will block the light,and the slits produced in them will cause serious diffraction of light after passing through the screen,which greatly reduces image quality on the sensor.It is of great significance to solve this problem to restore the clarity of the under-screen image.In this article,based on the theory of deconvolution,deep learning and image inpainting,the under-screen image is processed to be as close as possible to the up-screen image.Realization the imaging under the mobile phone screen is possible.The main content and innovations of this article are as follows:1.Deconvolution algorithm is applied to the image restoration of under-screen imaging of mobile phone.Calibrate the point spread function(PSF)of the front camera system of the mobile phone by fiber-coupled LED.To solve the problem of excessive noise in a single image,the traditional deconvolution method is improved by converting the color space of the image,deconvoluting the intensity channel to improve clarity,and filtering color channels to reduce noise.Non-local means(NL-means)is further used to process the image after deconvolution.This processing not only improves the signal-to-noise ratio of the image,but also obtains a better visual effect.It can be seen from the simulation results that the peak signal-to-noise ratio(PSNR)of the processed image can be increased by about 10%,and the structural similarity index(SSIM)can also be increased by about 0.2.2.Generative Adversarial Network(GAN)is introduced to the recovery of under-screen image.As shown in the simulation results that the peak signal-to-noise ratio of the degraded image can be increased by about 20%after trained GAN network.The dataset is constructed by the captured on/under-screen image pairs to train the GAN network.The experiment proves that the trained network can achieve a good recovery effect on the under-screen image.3.An inpainting algorithm based on Fast Marching Method(FMM)is introduced to deal with the saturated diffraction spots generated when capturing high-brightness scene.The repair mask is obtained by extracting the saturated position in the image,convolving it with the PSF.It can be seen from the inpainting result of the actual image that this method can successfully remove the saturated diffraction spots without affecting the overall visual effect of the image.