Research On Real-time Transmission And Management Technology Of UAV Freight Big Data

With the advent of the era of "UAV+",more and more logistics companies begin to pay attention to the application of UAV in logistics system.A revolution in subverting the logistics industry has quietly begun.However,if UAV cargo is to be put into use,there are still some technical challenges,such as real-time scheduling.Real-time transmission and management of large data of UAV cargo is the key link of cargo scheduling.Due to the characteristics of multi-source,massive and fast speed of UAV freight data,UAV scheduling has a high demand for real-time.In this thesis,the real-time data transmission,storage and processing technology of UAV freight big data is studied,and the real-time data transmission and management system of UAV cargo is designed and implemented according to the actual needs.The main works of this thesis are as follows:(1)Through the analysis of the existing real-time transmission,storage and processing technology of big data,the real-time transmission and management technology suitable for UAV freight big data is selected,and the related technologies and theories are expounded.(2)Aiming at the problems of HBase mass stream data storage,such as storage hotspots and high data storage load in a short time,a real-time storage optimization method for UAV freight stream data based on HBase is proposed.The method of this thesis optimizes the row key of UAV freight real-time streaming data,and adopts multi-source data buffer optimization and data server pre-partitioned data distribution strategy to solve data dispersion problems and data storage hotspots;At the same time,in order to solve the system overhead problem of thread life cycle,a multi-queue thread pool mechanism with dynamic priority switching is introduced to improve the processing performance of the system in high concurrent data state.Finally,a query and deletion method for the complex conditions of UAV freight big data is given to ensure the efficiency of data query.(3)Aiming at the problems that the Storm default scheduling algorithm has low utilization of node resources and long system response time,a dynamic load balancing strategy for Storm cluster based on cuckoo search algorithm is proposed.An improved cuckoo search algorithm is designed to simulate the task scheduling as the process of cuckoo nesting and spawning.The algorithm takes into account the real-time utilization of cluster resources such as CPU,memory,network bandwidth and so on.Through the optimization process of the cuckoo search algorithm,the node performance weights are adaptively determined,and the tasks are dynamically allocated according to the weights,which realizes the rational allocation of resources and the load balancing of the cluster dynamics,thereby reducing the cluster response time and improving the real-time processing of the system.(4)According to the actual needs of UAV cargo large data real-time transmission and management system,the system design is completed,and the optimization of data storage and processing is applied to the system.On this basis,the system framework and main functions are completed,and the preliminary verification shows that the effect is good.