Fisheye Distortion Rectification Using Deep Learing

Distortion rectification for fisheye cameras is a fundamental and important issue in computer vision and computer graphics.In order to obtain the visual range beyond the human eye,the unique design of the fisheye lens through its shape makes it have the advantages of a large field of view and a wide field of view coverage.But at the same time,the images taken with fisheye cameras also have severe visual distortions,and this distortion effect brings difficulties and challenges to their application in general vision tasks.Therefore,rectifying the distortion of fisheye cameras with high efficiency and high accuracy is the prerequisite and guarantee for the normal operation of subsequent vision tasks.However,because traditional rectification algorithms based on visual geometry usually require specific landmarks or known cameras,when processing a single fisheye image captured by an unknown type of camera in unknown environment,its rectification ability is very limited,and the corrective effects are usually unsatisfactory.In order to overcome the disadvantages of the existing distortion rectification algorithms,this paper uses Convolutional Neural Networks(CNN)to re-understand the distortion correction problem.Instead of using traditional visual features,the CNN's strong representation ability is used to regressively learn input fisheye images.The distortion parameters can straighten the "distorted lines" that are curved due to the fisheye projection,thereby achieving effective distortion rectification.This article has made the following contributions:First of all,we propose a new exploration direction for correcting fisheye distortions.In the process of learning how to straighten the "distorted lines" in fisheye images,the fisheye image distortion rectification is realized.It has been proved through experiments that the application of the traditional calibration algorithm theory in the framework of deep networks: "the ‘distorted lines' generated by fisheye projection should be straight lines in normal perspective images" is promising and meaningful.Secondly,we designed a novel end-to-end deep network framework.Aiming at the non-linear characteristics of the distortion distribution of fisheye images,a multi-scale perceptual learning module was designed to balance the local and global distortion effects.In response to the introduced errors caused by the "distorted line" detection,uncertain regularization was designed to further improve the network performance.Finally,based on a large-scale fisheye dataset that is not yet public,we created a fisheye database with corresponding distortion parameters and line labels.A variety of different evaluation experiments are performed on this database,and the effectiveness and advancement of the distortion rectification network proposed in this paper are verified.