Research On Remote Sensing Algorithm Program Integration Based On Docker Container Technology

Remote sensing data have obvious big data characteristics owing to the continuously improving resolution of spatial,temporal,spectral,and radiometric for remote sensing data also increases the data type.When using traditional methods for development integration,it is difficult to achieve effective integration.Because different algorithm programs may be written by different people,the development language and environment are diverse and the configuration is complex.It will affect the efficiency of integration,if unified into a specific locale.Excessive resources are consumed when the system is deployed.This is not conducive to efficient and rapid integration and deployment.This is not conducive to the rapid industrial application of remote sensing data.The secondary development environment of remote sensing image processing software such as ENVI is single.Remote sensing cloud service platforms such as Google Earth Engine provide online customized remote sensing information product production services.But individuals need to learn to use these APIs to write their own algorithm scripts to use these cloud computing platforms.They do not meet the plugand-play requirements of the program.Therefore,in this paper,we design and realization a system framework for rapid integration of remote sensing algorithms on the basis of Docker container.The system supports different programming languages and operating environment differences,the scheduling of different production processes and data distributed storage and distributed efficient computing.The framework consists of an automated image encapsulation mechanism for remote sensing algorithms,a unified image distribution management,a containerized orchestration service for the production process of remote sensing information product,and a container scheduling scheme about daemons.The programmer then uploads the program code or executable and the Dockerfile to Gitlab for code hosting.The Pipeline workflow plugin is used in Jenkins for continuous construction of images.We designed a container operation scheme based on JAVA platform.It can realize containerized operation of specific remote sensing products.Finally,a Swarm container cluster environment and GlusterFS distributed file system were created.They serve as a container operating environment and a remote sensing data storage and management environment.In this paper,the typical remote sensing index product production of Landsat data and the RPC correction of GF1 data are taken as examples to carry out systematic verification.We performed computational and deployment performance experiments on NDVI programs in Docker virtualization environment and physical machine environment.We observe nearly no performance difference running an application in a Docker container virtualization environment is basically consistent with the physical machine environment in terms of system load metrics such as operational efficiency and memory footprint.The significance of the above research includes the following aspects in this paper.Firstly,we encapsulate remote sensing algorithm programs in different development language environments.This does not require consideration of the environmental differences of the various algorithmic programs.For remote sensing algorithm researchers,this method is beneficial to the reuse of remote sensing algorithm programs and the sharing of processes.Secondly,for users of remote sensing algorithm programs,only the container environment needs to be installed during deployment.Finally,we use container clusters to produce remote sensing information products,which can save physical resources and improve processing efficiency compared to virtual machines.