Resolution-Adaptive Image Hybrid Distortion Correction Method Based On Improved U-Net Network

Computer vision is a scientific technology that uses cameras and other devices to capture and analyze image information.It has a wide range of applications in fields such as 3D reconstruction,video surveillance,and the automotive industry.However,due to the optical characteristics of the camera lens,the images captured often exhibit geometric distortion,which can reduce the accuracy and quality of the image,and thus affect the subsequent analysis and processing of the image.The elimination or reduction of such distortion and the geometric correction of images have important application value in the field of computer vision,image processing,and other related areas.The current methods for correcting image hybrid distortions have some problems and limitations.Traditional math-model based methods rely on prior knowledge of the distortion type and distortion parameters.However,due to the diversity of sensor types,complex imaging environments,and complex distortion types in images,it is difficult to describe distortions using a fixed mathematical model.Therefore,the applicability of these methods is limited.Although deep learning methods can perform distortion correction in a data-driven manner,there are also some limitations.On the one hand,deep learning methods rely to some extent on datasets and often can only handle specific types of distortion.They do not perform well with hybrid or unknown distortions.On the other hand,deep learning methods are limited by network architecture and it is difficult to adaptively adjust the size and quality of input images.Based on the above research background,we propose an resolution-adaptive Image hybrid distortion correction method based on improved U-Net network.The specific contents are described as follows:(1)To address the problems of insufficient quantity and single distortion type in existing datasets,we created a hybrid distortion image dataset.First,images containing linear structural features were selected from publicly available image datasets,and then synthetic chessboard images were merged as the initial dataset.Multiple sets of distortion parameters were obtained through camera calibration work,and radial and tangential hybrid distortions were added to them based on a polynomial distortion model to generate a hybrid distortion image dataset.(2)To improve the correction accuracy of hybrid distortion images,we propose an image hybrid distortion correction method based on improved U-Net network.This method transforms the distortion image correction problem into a problem of predicting the coordinate displacement of pixels in the distorted image.First,the U-Net network was improved using spatial attention mechanisms based on the geometric features of the distorted image.This enhancement strengthens the network’s ability to extract image features and focus on the distorted areas and edge details.Secondly,we designed coordinate difference loss and image resampling loss functions,and optimized the network model training by imposing dual constraints on the coordinate positions and image quality.Finally,ablation experiments were conducted on the dataset constructed in our paper to verify the effectiveness of each module in this method.Compared with recent deep learning-based distortion image correction methods,the comparative experiments showed that this method outperforms others in both quantitative metrics and subjective evaluations.The average absolute error of spatial coordinate correction for distorted images was 0.2519.In addition,our method was applied to perform calibration experiments on the optical images acquired by Go Pro cameras.The experimental results further verified that our method has good generalization ability.(3)The image distortion correction method based on the improved U-Net network has limitations that it can only correct images with a fixed resolution.To address this issue,an adaptive resolution image distortion correction method has been proposed to achieve correction of distorted images with arbitrary resolutions.While retaining the core framework of the U-Net network,an image downsampling process has been introduced,allowing the network to adapt to inputs of different resolutions.In the output section,the predicted coordinate difference matrix is upsampled and combined with the input image to obtain a distortion-corrected image with the same resolution as the input image through resampling.To ensure the accuracy of distortion correction,the contour extraction and Hough transform methods are used to extract line information from the corrected image.A line-structured loss function is also designed based on the extracted lines to calculate the average angular deviation and average distance deviation between the corrected image and the label image.A distortion dataset containing images of different resolutions(352～1100 pixels)was constructed for the improved network,and the network was trained using this dataset.Comparative experimental results between the two methods,before and after improvement,show that the improved method has a significant advantage in correcting multi-resolution distorted images.Furthermore,the method has good correction effect on distorted images of resolutions outside the training dataset,demonstrating the adaptive image resolution ability of this method for hybrid distortion correction.There are significant implications for improving the quality and accuracy of images,as well as enhancing the analytical and processing capabilities of images,through the correction of geometric distortion in images.The hybrid distortion correction method proposed in our paper effectively corrects images with different resolutions,achieving the correction of multi-resolution hybrid distortion images and avoiding the complex calibration and computation processes in traditional correction.Additionally,we also constructs an image dataset that includes hybrid distortion types with different resolutions,providing a method for researchers to generate hybrid distortion images for future studies.