Research On Key Technologies In Radio Astronomical Data Realtime Processing

Data is the result of astronomical observations and the core basis for conducting astronomical research.With the development of astronomical equipment and observation technology,a new generation of astronomical telescopes has emerged,and the ability of astronomers to acquire astronomical observation data has been unprecedentedly strengthened.Astronomical data has arrived in a real-time,sequential,massive and infinite way.How to process observation data in real time through distribution and parallel means that astronomers face unprecedented challengesThis paper studies the problems faced by the Yunnan Observatory's 40-meter radio telescope pulsar digital terminal and the real-time data processing of new generation cm-decimeter aperture telescope,MingantU SpEctral Radioheliograph(MUSER).The detailed work description is as follows(l)Real-time data acquisition is the starting point of most real-time processing pipelines for astronomical data.This study analyzes the characteristics of current radio astronomy data stream transmission,and uses kernel bypass and zero-copy user-space network acceleration technology to solve the performance of traditional Linux operating system network protocol stack.This paper realizes the lossless data acquisition of line rate under 10 Gigabit Ethernet.The data acquisition framework based on this technology has been successfully applied to the data acquisition of the digital back end of the YNAO 40-meter pulsar(2)In addition to the use of dedicated hardware chips,the central processor's traditional solution to deal with astronomical massive data,due to excellent floating?point performance and better cost performance,the use of graphics processing units(GPU)for real-time data processing has become an inevitable trend.A real-time massive data processing framework based on GPU was designed.The framework is applied to the real-time data processing of coherent dedispersion,which realizes decoding,transposition,Fourier transform,dedispersion,subchannel,inverse Fourier transform,polarization detection,folding,denoising,and archiving.The data processing module was tested on YNAO 40-meter telescope.In addition,for the radio frequency interference problem faced by the 40-meter telescope pulsar observation,the mixed signal is decomposed by independent component analysis,and the independent RF interference signal and pulsar signal are decomposed according to the distribution characteristics of the pulsar signal and the interference signal in the observed signal.(3)Real-time processing needs to make full use of all computing resources to achieve seamless operation and fast migration of astronomical computing software in a heterogeneous environment,which can improve the utilization efficiency of computing resources.We have studied the application of OpenCL technology in real-time astronomical data processing in heterogeneous environments.Based on this,MUSER's imaging gridding algorithm and cleaning algorithm are implemented,which guarantees that the algorithm's operating efficiency is basically unchanged.It is no longer limited to NVIDIA's GPU environment,providing further scalability for real-time data parallel processing in heterogeneous environments.(4)Real-time processing of data processing horizontal expansion computing capability on a cluster using a distributed system is also a maj or research content.In order to improve the efficiency of flexible deployment and automatic expansion of distributed real-time computing environment,the astronomical real-time processing cluster agile construction and deployment based on lightweight container Docker technology is studied,and the MUSER existing system is encapsulated by container technology,and in different hardware.The performance is compared with the physical machine and other virtual machine technology.The paper studies several problems in the real-time processing of massive astronomical data.The data acquisition technology based on the user space can apply the data communication of the super high IO at the front and rear ends of the digital terminal.Data processing based on GPUs and OpenCL with heterogeneous computing platforms can greatly help accelerate computation and can enable better,faster science.The virtualization of lightweight containers makes it easy to build real-time computing clusters.The research content provides a reference for the real-time processing of astronomical data,laying a good foundation for the next step of related work.