Deep Learning For Rectification Of Fisheye Image

With the increase of requirement on information acquisition,as an ultra-wide-angle camera,fisheye camera is more and more widely used in different areas.However,non-linear radial distortion is introduced while fisheye cameras benefiting the information acquisition.Therefore,the rectification of fisheye image is a crucial process in reality applications.Recently,rectification algorithms with high accuracy based on traditional computer vision have been developed.However,various fisheye cameras have different structures and principles which limit the scope of applications of rectification algorithms.To counter the problems above,a method for rectification of fisheye image based on deep learning is proposed.The proposed method which can be applied in various contexts breaks the limitation that different algorithms should be used for different types of fisheye lenses.In the proposed method,convolutional neural network extracts distortion information of fisheye image and learn the mapping between distortion features and its corresponding rectification parameters.In order to train the proposed neural network,a fisheye image dataset is synthesized which contains 2500 images captured by pinhole cameras,25,000 fisheye images which are generated by adding 10 kinds of fisheye distortions on each undistorted image and their corresponding rectification parameters in various context with different distortions.During the training of neural network,the rectification parameters estimation network is trained independently to get an initial solution.Then it is fine-tuned by comparing the difference between rectified image and its corresponding undistorted images.The trained network predicts the rectification parameters accurately and performs the rectification of fisheye images very well.The experiment results indicate that,the proposed method performs the rectification of fisheye image very well without any known parameters of cameras.Furthermore,the proposed method breaks the limitation on scene or type of fisheye camera in existed algorithms which are based on traditional computer vision.