Research On Fisheye Camera Based 360 Degree Panorama Driving-assist System

Since 2013, VR(Virtual Reality) has been rapidly developed. The game companies and the biggest Internet companies have made contribution to the development of virtual reality. Facebook purchased a company that makes virtual reality headsets, Oculus VR, for $2 billion, which indicates the bright future of VR. Google provides Street View service, and it also features a stereoscopic 3D mode. The main research domain of this paper is fisheye-image-based virtual reality. The specific descriptions are illustrated as follows:(1) Fisheye camera calibration and fisheye-image correctionWe use two methods for fisheye camera calibration which are Zhang's technique and omnidirectional camera calibration toolbox, respectively. For deficiencies of Zhang's technique and omnidirectional camera calibration toolbox, in this paper, we propose an equidistance-projection-based calibration method for fisheye lens by using checkerboard. We implement three algorithms, i.e. perspective projection based, stereographic projection based and image content based, to undistort fisheye image.(2) Fisheye image based virtual reality technologyWe make researches and analysis on 360-degree camera, and spherical panorama stitching algorithm is considered as the most important part of fisheye image based virtual reality technology. Spherical panorama stitching techniques including image preprocessing, feature extraction and the generation of matching point, image registration, bundle adjustment and panorama output. For image registration, we ignore the transformation of cameras.(3) 360-degree omnidirectional image based driving-assist systemsWe use two methods for establishment of hardware platform which are car with 360-degree camera and car with four fisheye cameras in four different orientations. Different from the bird-view driving-assist system, the users can observe the scene from 360-degree omnidirectional image freely, not just overlook. This system processes the multimedia streams from the four fisheye cameras in four different orientations in real time in order to obtain the omnidirectional image. Finally, with the way of human-computer interaction, 360-degree omnidirectional image based driving-assist systems will show a 3D scene to the driver.