Research On Calibration Method Based On Big Data And Its Preliminary Application

With the continuous increase of the number,function and type of satellite sensors,the complexity of calibration task continues to rise.Therefore,traditional calibration methods and technologies are increasingly difficult to meet the needs of multi parameter,high efficiency and high frequency calibration,as well as to improve the efficiency and frequency of calibration.Incorporating with the application of new calibration methods such as global calibration field network and automated calibration,this dissertation the technique to realize multi-source,high efficient and high frequent calibration process.The domestic and foreign calibration multi-source data storage and management methods are investigated,and big data technologies such as distributed file storage system,distributed database and distributed computing are introduced to establish big data calibration database,which provides time continuous calibration basic parameters for high-frequency calibration,increases the observing oppotunities of remote sensing satellite over the calibration sites,and improves the frequency of in orbit radiometric calibration.Based on Hadoop system,this dissertation proposes a prototype system framework for big data processing,The system realizs the function of distributed storage and distributed computing,to guarantee the calibration ability and calibration quality of remote sensing data.,HTML5+Javascript+React to build the front-end page of big data calibration platform,Java language+spring boot are used to build its back-end services,and the SWIG is used to call C/C++library,GDAL library,HDF4 library,and HDF5 library.Finally data and calibration services based on the B/S architecture and SaaS-like are implemented.In order to increase the time continuity of the calibration parameters such as surface and atmosphere properties,this dissertation designs automated climbing function to obtain multi-source calibration data which is similar to network reptile.This function realize acquisition,pre-processing and storage of the calibration multi-source data such as water vapor,ozone and surface,and provides stable data support for long-term sequence calibration.Thise technique significantly reduce artificial measurements of these parameters at calibration sites.The absolute radiometric calibration function of the big data calibration platform in this thesis can automate calibration test of the S-NPP VIIRS between April 2018 and December 2018,18 effective calibrations with the relative deviation less than 5%of each band have been made.The results showed that the apparent reflectance obtained respectively by the site automated observation and the star measurement have good consistency.At the same time,the root mean square of 18 site automated calibrations is less than 2.7%which shows that it can be used for the satellite's high-frequency on-orbit radiometric calibration,as well as detection of operation status and trend.For GF-1 WFV3,15 calibration sites in China have been used for more than 250 long-time calibration tests during 2013-2018,and about 100 effective calibration results have been obtained.Compared with the results of automated calibration in 2018,the relative deviation of the four channels corresponding to the results of the two calibration methods is within 1%,which shows that the two calibration methods have good consistency and improve the reliability of the calibration results,These results indicate a promising application feasibility of the new calibration procedure realized in this dissertation.