Research On Key Technologies Of Cloud Platform For Large-scale Signal Deep Search

The use of space electromagnetic signals such as satellite and broadcasting to detect and search the target airspace provides a new feasible means for discovering invading aircraft.However,in the process of comparing the pure signal with the reflected signal,a large number of signal processing operations,such as fast Fourier transform,bring many challenges to the back-end data processing,such as large data volume,high real-time requirements,and high reliability requirements.Traditional computing models are no longer sufficient.In order to provide faster,more convenient,stable and efficient computing services,this paper improves the data processing capability of large-scale signal search cloud platform from three aspects:parallel algorithm design,cloud platform construction and computing task scheduling:(1)A Spark-based parallel signal depth search algorithm is designed.Aiming at the problem that the traditional algorithm stores limited data and reads data slowly,this paper is based on the MapReduce method,stores the data in the elastic distributed data set in the Spark framework,distributes the computing tasks to each computing node,and completes the parallel fast Fourier transform as the core.Signal data processing.The system data is increased by distributed storage data,and the intermediate data is stored in the operation process to improve the signal search speed.(2)Designed to realize a containerized cloud computing platform for signal depth search.In order to improve the utilization of computing resources and simplify the development process,this paper adopts the container technology to realize the virtualization of computing resources,and utilizes its lightweight,convenient deployment and easy to expand features to quickly deploy lightweight computing cores that are lighter than virtual machines.In the aspect of container management,the Kubernates container management system is used to manage the deployment development and maintenance management of the cluster,and the computational resource virtualization is realized by programming the Docker template to provide stable and powerful computing capabilities.In order to better serve the signal depth search application,the GPU device is integrated into the cloud computing platform to perform the fast Fourier transform task,which improves the processing speed of the algorithm.(3)In order to make better use of computing resources and serve big data processing tasks,this paper designs a task scheduling algorithm based on machine learning.When the computing tasks of the cloud platform are processed,the utilization of each host resource is uneven.The cloud platform is used to monitor the cloud computing system in real time,and the usage of resources such as CPU,memory,and storage in the host is obtained.According to the use of these resources,find the host with too high resource usage and too idle,and implement the optimal scheduling of the container through the relevant scheduling algorithm.In the task scheduling aspect,the machine learning method is used to analyze the running status of the host and the container.The EM algorithm is used to extract and classify the similar containers,and the number of idle and overloaded hosts is found through the migration of the container,which improves the resources while saving.System stability.