Research On Classification And Target Detection Of Spherical Image Based On UAV Platform

In recent years,driven by the global market background of "industry 4.0" and "made in China 2025",machine vision that simulates human vision to achieve object recogni-tion,tracking,detection and other tasks has been widely used.It covers military,medical,industrial,aerospace,and scientific research fields.As a brand-new machine vision so-lution,the aerial photography system based on the Unmanned Aerial Vehicle platform has received widespread attention in recent years.Because the combination of UAV and cameras can give full play to the advantages of UAV,such as convenient operation,strong environmental adaptability,through which we can realize real-time and efficient vision tasks such as target recognition and detection.It is worth noting that most of the current airborne camera solutions use traditional perspective cameras as imaging platforms.Due to the limitation of the inherent viewing angle range,there will inevitably be blind spots in imaging.This may cause immeasurable losses in areas that require high environmental information accuracy and comprehensiveness,such as military and surveillance.Therefore,we propose a UAV machine vision solution that uses a fisheye spherical camera with a 360-degree viewing angle instead of a camera equipped with a traditional optical lens in this thesis.This solution takes full advantage of the large viewing angle of the fisheye lens.Compared with the image taken by a traditional camera,it can contain more data information.Compared with the splicing of multi-plane lens images,it sim-plifies data collection and avoids using a large amount of hardware resources,which has a wide range of application scenarios.However,although the fisheye spherical camera is utilized to obtain a larger viewing angle,it also inevitably introduces image distortion.Therefore,based on the UAV machine vision platform equipped with a spherical camera,distortion processing methods of fisheye image,and deep learning technology,this the-sis has made relevant research on the spherical image classification and target detection under the UAV platform,mainly including the following work:(1)This thesis first introduces the imaging principles of traditional perspective lenses and fisheye lenses,including their corresponding optical models.At the same time,this thesis describes the projection model of the fisheye lens in detail,and through the com-parison of traditional camera images and fisheye images,we analyze the causes of fisheye distortion and traditional correction methods.Also,in order to achieve image classifica-tion and target detection based on "UAV+360° panoramic camera",we also introduced the theoretical basis of neural networks,focusing on analyzing the principle and appli-cation of convolutional neural network(CNN)in image processing,and we analyze and summary the characteristics of the classic network model in the development of CNN.(2)In order to realize the research of spherical machine vision based on the UAV platform,this thesis uses the E360 multi-rotor UAV of Zhongke Haodian and the high-definition fisheye camera THETA V of Ricoh to build the corresponding UAV spher-ical vision hardware platform.Based on this platform,we have established a spheri-cal panorama dataset SWU-dataset for scene classification within Southwest University,which mainly includes the school’s Dongfanghong Conference Hall,School History Mu-seum,Gymnasium,Banyue Building,and other landmark locations.Based on the SWU-classify,we analyzed some problems in the airborne spherical lens imaging scheme and proposed a corresponding image preprocessing method.In addition,we proposed a scene classification method based on discrete spherical images and compared it with the exist-ing spherical image projection methods(VGG,ResNet,EfficientNet),and analyzed the performance of the algorithm under noise and image rotation(caused by drone vibration).(3)In addition to the related exploration of "UAV+360° panoramic camera" in the scene classification task,we also established a spherical panoramic image dataset SWU-detect for target detection based on the UAV platform.And research on related target detection tasks is carried out.First,we analyzed the performance of two-stage Faster R-CNN and one-stage SSD target detection algorithms in spherical image target detection.Secondly,we also analyzed the performance of the current best comprehensive perfor-mance YOLOv5 algorithm in corresponding detection tasks.In particular,considering the geometric distortion introduced by the spherical image plane representation,we have made a corresponding improvement to the YOLOv5 algorithm based on deformable con-volution.The simulation results show that our proposed improvement scheme applied to the target detection task with geometric distortion can obtain a good optimization effect.