Research On Speckle-autocorrelation Non-Line-of-Sight Imaging Technology Based On Deep Learning

Non-line-of-sight(NLOS)technology restores the shape and reflection information of the target hidden outside the line of sight of the sensor by analyzing the multiple scattered light of the hidden target in the occluded area,this technology has good application prospects in the fields of machine vision,remote sensing observation,medical exploration and intelligent assisted driving.Most of the existing NLOS imaging systems need to be equipped with high-sensitivity time-resolved single photon detectors and ultrafast pulse lasers.However,such methods are highly dependent on expensive hardware equipment,which requires a lot of time for data acquisition in the imaging process,and the complexity of reconstruction algorithm is high.Therefore,the wide popularization and application of this technology is still a challenge.To solve the above problems,this thesis studies NLOS imaging method based on laser speckle correlation,recovers the hidden scene from the easily obtained speckle pattern,and designs the reconstruction algorithm based on deep learning,and finally realizes the low-cost,real-time and robust NLOS imaging.The methods and innovations proposed in this thesis are as follows:(1)A real-time NLOS imaging algorithm based on deep residual neural network is proposed.Due to the lack of real-time and economy of time-of-flight imaging method,the practical application of NLOS imaging is seriously limited.From the perspective of data acquisition methods and reconstruction algorithms,this thesis restores the hidden object through the easily obtained laser speckle pattern,establishes the laser speckle autocorrelation NLOS imaging noise model,generates synthesize the training dataset,construct the deep residual neural network,and learn the autocorrelation mapping from the speckle autocorrelation image to the hidden target.The experimental results show that the deep residual network model realizes real-time NLOS imaging on synthetic dataset,simulated autocorrelation image dataset and practical autocorrelation image dataset.Under the exposure time of 1sec to 1/128 sec,the deep residual network method can recover the shape of the hidden target at the speed of milliseconds,and the comprehensive performance of the network model is better than the comparison method.(2)The speckle autocorrelation NLOS imaging algorithm based on Generative Adversarial Network(GAN)is proposed.Although the deep residual neural network algorithm realizes real-time NLOS imaging,however,in view of the degradation of the network model,the accuracy of recovering the details of the hidden target from the practical measured speckle autocorrelation image is poor.In this thesis,a speckle autocorrelation NLOS imaging algorithm based on GAN is proposed,and a conditional supervision mode is added to learn autocorrelation mapping with higher accuracy.The experimental results show that GAN model not only significantly improves the PSNR and SSIM indexes of the prediction results of the synthetic dataset compared with the comparison method,reaching 19.7174 d B and 0.8438 respectively,but also recovers better visual performance than the comparison methods such as deep residual network,UNet and UNet++,especially in the task of recovering complex targets,it can accurately recover targets with complex shapes such as “N” and “S”.