Study On The Data Mining Technology Of Remote Sensing And Unmanned Aerial Vehicle Low Altitude Remote Sensing Image Based On Distributed Systems

The multidimensional,multiscale,high-resolution remote sensing image data and the unmanned-aerial-vehicle(UAV)low-altitude remote sensing image data are increasing rapidly with the advancement in the application of the big data technology in space remote sensing.However,the traditional classification software has some limitations,such as long processing time and excessive fluctuations in the classification accuracy,when processing such massive amounts of image data.This thesis designs a set of Flink-based fast UAV remote sensing image classification systems by combining the Flink stream processing operation architecture and the Hadoop distributed system and optimizing cluster overload protection and data skew prevention to improve the classification efficiency when the classification accuracy fluctuates within a rational range.Because of the large amounts of data and the complicated information associated with the existing remote sensing images and the UAV low-altitude remote sensing images,this thesis introduces Flink distributed stream processing into the remote sensing image data classification process.To prevent problems related to asynchronization,coupling,and thread safety in multithreading,this thesis uses Flink cluster to read images from the Hadoop distributed file system for stream processing,which is advantageous when compared to traditional processing with respect to the reduction of the reading duration and unnecessary I/O consumption.Further,this thesis proposes an optimization method by improving the checkpoint mechanism of Flink cluster with Kafka;in this method,the faults observed when processing huge volumes of images are temporarily stored in the Flink cluster to eliminate overload while interactively processing huge volumes of images based on clients and server clusters.By determining whether the server operation reaches its upper limit of performance when processing mass remote sensing images and UAV remote sensing images in the dual channel processing mode,the Kafka distributed message subscription system will activate to achieve caching after encrypting and compressing images at the time of reaching the upper limit of cluster processing.This prevents the reduction of the cluster processing efficiency and file corruption owing to data backlogging.If the cluster is not subjected to pressure,it will be suitable to process the remaining data obtained from Kafka.When compared with the traditional Flink cluster,this mechanism can reduce the faults in image processing to a certain extent and increase the upper limit of cluster processing.This thesis optimizes the Lib classification algorithm library function of the Orfeo Tool Box using the KeyBy and Reduce mechanisms of Flink after the extension of the mass data of the remote sensing images to improve the efficiency of remote sensing image as well as UAV remote sensing image processing and prevent data skew during the classification of the Flink cluster.In addition,this thesis proposes a classification optimization method for the UAV remote sensing images based on the Flink cluster.In case of mass remote sensing image processing,this method ensures higher classification efficiency and greater system extensibility than those provided by the traditional classification method based on the high image classification accuracy.To evaluate the validity of the aforementioned mechanism and the classification optimization method as well as the concurrent ability of the combined system,this thesis designs a distributed fast classification system for remote sensing images and UAV remote sensing images based on Hadoop and Flink,involves performing tests 1 under the developed system environment,and presents several groups of comparative experiments.According to the experimental results,the proposed optimization method increases the overall classification efficiency by approximately 35%and reduces the failure rate when processing by 12%in case of mass remote sensing image processing compared to the traditional classification method.Furthermore,in case of UAV remote sensing image processing,the overall classification efficiency increases by approximately 17%and the failure rate during processing decreases by 15%.