Real-Time Object Detection For Panoramic Images

The panoramic image,or the 360-degree panoramic image is a kind of 2D image that is captured with 360-degree panoramic camera and contains the visual information from every direction.The object detection is one of the most important research fields in computer vision,whose target is to find the objects in an image and recognize those objects.The object detection for panoramic images is widely applied in many areas such as automatic driving,navigation of drones and driving assistance.It can offer guidance,assistance and caution to drivers and improve the safety and driver experience.The panoramic image can be considered as a 2D image which is the result of a 360-degree panoramic sphere being expanded along a longitude line,like the map of the earth.Therefore,pixels are stretched and objects are twisted when they are close to the poles.Existing state-of-the-art approaches for object detection are all based on convolutional neural network.The detecting ability of these kinds of methods depends on the training sets.However,there are no existing datasets of panoramic images for object detection.Thus these approaches cannot work well in detecting objects in panoramic images.Additionally,some objects in a panoramic image may be divided into two parts by the longitude line,along witch the 360-degree panoramic sphere is expanded.One part of this kind of objects lies on the left of the image and the other part lies on the right.Existing methods of object detection will be misled to detecting those two parts as two different objects or detecting only one part of them.Aimed at the problems that are mentioned above,this paper proposed a systematic solution based on convolutional neural network to solve the problems in detecting twisted objects and divided objects.The main contributions of this paper are concentrated in several aspects as follows.Firstly,a strategy of reusing the existing ordinary training sets is proposed in this paper to detect twisted objects in panoramic images,instead of establishing a huge panoramic image training set manually.Secondly,a duplication stage and a fine-tuning stage are proposed.The duplication stage can produce extra detecting information for the finetuning stage.The fine-tuning stage can fine-tune the detecting results with these extra information,and particularly,merge divided objects.Thirdly,the duplication stage and the fine-tuning stage are designed low-coupled with the convolutional neural network,so that the convolutional neural network can be replaced with any other convolutional neural network.In the experiment,contrast experiments are adopted among existing methods and the approach proposed in this paper.The experiments compare the cost of time and the detecting results of these methods.The experimental results show that the proposed method for detecting objects in panoramic images can reach 80% accuracy and 75% recall rate,and meanwhile achieve real-time.Additionally,the argument of rationality for reusing the dataset is presented,which guarantees the method to reuse the existing training set being rational.