Research On The Development Of BTOPMC/SCAU Distributed Basin Hydrological Model System

BTOPMC（Block-wised TOPMODEL with Muskingum-Cunge method）is a grid-based distributed basin hydrological model.In BTOPMC,the large basin is divided into a serial of blocks or subbasins according to the basin topographic features,soil types,land covers and climate patterns.By applying TOPMODEL（TOPographic MODEL）to each subbasin,runoff generation is modelled at grid cells.Flow routing is concentrated to each subbasin outlet in model calibration phase,or to the basin outlet in modelling phase,using an adaptive Muskingum-Cunge method.It is a parsimonious and physically sounded model with a flexible architecture to be extended and to include more modules.The topographic information needed for runoff generation and flow concentration can be easily derived from basin DEM（Digital Elevation Model）.It has been widely applied to model many basins in Southeast Asia.With the improvement of its runoff generation and flow routing processes by applying the double-source model of Shuttleworth-Wallace for estimating basin potential evapotransipiration,the new river-width model and SCE-UA global optimization algorithm for parameter calibration,BTOPMC is advanced to a function fully and sophiscated distribued basin hydrological model,renamed BTOPMC/SCAU（South China Agricultural University）model.At present stage,it is urgent to be systemized as a software for its further development,promotion and more application.With this passion,a model system of BTOPMC/SCAU is developed using modern technology of computer,network,database and visualization according to software engineering principles.In the hydrological modelling of Hanjiang river basin,all functions of the system have been demonstrated well.The system has 5 key features:1)User friendly interface,wizard navigation menu,simple user interface design and easy to use.2)The computational efficiency of the model is improved by grid grading parallel computing based on sub-basin sequence.3)It creates parameters automatically,saves model running time and improves accuracy.4)It is easy for future maintenance and expansion by multi-layer structure and inter-layer decoupling design.5)It allows multi user access by using client/server structure.The main contents and results of the research are as follows:（1）According to the topology of basin water flow,a parallel flow calculation method based on sub-basin order and grading at the grid level is proposed.The basin is divided into several sub-basins based on the distribution of hydrological sites,and the sub-basins are arranged from small to large by their accumulation area.For example,the smallest sub-basin is identified as‘1’;the largest will be‘m’.In automatic calibration（the most time-consuming procedure）,it starts the smallest sub-basin,and gradually progress to larger sub-basin.Sub-basins are either in parallel or coupling relation.When the model calibration reaches a coupling hydrological site,the upstream sub-basin calibration is automaticly completed and there isno need to repeat since the area of those coupling sub-basins is always less than area of the target hydrological sites.The calibration only happens between grids of sub-basin.It improves efficiency.For grid,it can be classified by theiraccumulation area.For example,if theaccumulation area is a one grid unit,it must be the source grid of the sub-basin.It does not get water flow from other grids but itself.Then it is grid class one.If the area is a two grid unit,it gets water flow from one upstream grid besides from itself,so it is grid class two.Class 3,class 4 and etc.is identified in thesame way.The same level of grids can be calculated in parallel because there is no hydraulic connection,no water exchange.For example,the number of class one grid in the i-th sub-basin is N₁,the available thread is P₁.If user uses s₁ threads（s₁≤N₁及s₁≤P₁）,then the grids for each thread are N₁/s₁.Once class one grids finish automatic calibration,it goes to class two grids,class three grids and so on.By thesame methods,calibration finishes from one sub-basin to thenext sub-basin until the whole basin area.The above parallel computing algorithm is implemented by OpenMP（Open Multiprocessing）programming and verified in Hanjiang river basin study.Flow concentration is simulated in 9 hydrological sites in the Hanjiang river basin at 1km grid scale.The server used for testing is the DELL T7810workstation,the operating system is the ubuntu16.04 server,the specific hardware configuration is as follows:double Intel Xeon E5-2650 v4 12 cores,total 24 cores,2.4G cpu clock,32 G RAM and 1T hard disk.When 4 cores are used for the simulation,the result shows the system efficiency is improved by 2.8 times.When 20 cores are used for the simulation,the result shows the system efficiency is improved by nearly 12 times.（2）Based on the SCE-UA（Shuffled Complex Evolution developed at University of Arizona）,a global optimization algorithm,and the parameters of the BTOPMC/SCAU model is automatically calibrated by parallel computing.Automatic calibration does not need human intervention.Users do not have to have experience and expertise of hydrological knowledge.But the model does need fast,effective of the objective evaluation functions.The BTOPMC/SCAU model system provides 7 objective evaluation functions.They are simple least squares error,time error minimum error,logarithmic error minimum error,balance time error minimum error,heteroscedasticity maximum likelihood estimation,Time-varying variance maximum likelihood estimation and the geometric mean of the Nash efficiency coefficient and the water balance error of each hydrological process.The system provides default parameter for the SCE-UA algorithm,as well as manual parameter input and modification.In Hanjiang River Basin study,default parametersand the geometric mean of the Nash efficiency coefficient and the water balance error of each hydrological process are used.Automatic calibration function is verified in this study as well.（3）BTOPMC/SCAU model is softwareizedand systematized.A multi-layer architecture is designed for BTOPMC/SCAU model system,which makes the system easy to expand.The system has five layers from bottom to top.The 5 layers are data management,models,communication,data illustration and user operation layer.The upper layer relies on the lower layer and the lower layer supports the upper layer.Data management layer lays at the system base,responsible for data storage and maintenance.Data is divided into two types,non-structured data and structured data.Non-structured data includes input data such as terrain,soil,vegetation,remote sensing,and the data generated during the operation of the model,organized and stored in files and document format.Structured data is mainly measured rainfall,meteorological and runoff data organized and managed in the relational database format.The model layer is responsible for calculation/computing.It includes terrain sub-model,potential evapotranspiration operator sub-model,runoff sub-model,and flow sub-model.All modules are programmed by C.The communication layer is the communication bridge between the layers,mainly responsible for the message transmission between the completion of heterogeneous systems and the data transfer between user operation layer,data expression layer and the databases.The message transmission uses C and Java programming based on Socket mechanism and the data transfer uses JDBC driver.Data illustration layer is to visualize all kinds of data by graphics and tables,programed by Java.The user operation layer uses the wizard menu and the user friendly interface to simplify the hydrological simulation process of the river basin,programed by Java as well.The Client/Server mode is designed in BTOPMC/SCAU model system.Data illustration layer and user operation layer codes are deployed in the client;Data layer and model layer codes are deployed in the server.The client and the server complete the message by the communication layer.（4）In order to adapt the model to network operation mode and make full use of the high performance computing and large-capacity storage,a high concurrent information processing model was designed base on HH（Half-Synchronize/Half-Asynchronize）pattern.The information concurrent processing model has 3 layers,The layers are asynchronous task layer,queueing layer and synchronous task layer.In order to meet the client connection request quickly,and to process the hydrological simulation business of all clients correctly,the communication function of the server is separated from the model business function,the asynchronous task layer is responsible for communication,and the synchronization layer is responsible for business.The asynchronous task layer is responsible for responding quickly to client requests and does not handle any model business.The asynchronous task layer focus all communication tasks in the main process itself,to avoid complex multithreaded and synchronization,and save the huge system overhead caused by frequent switching between multithreading/processes.The main process converges multiple multiplexer basede on using asynchronous communication mode,it can process hundreds of client connections simultaneously.The queuing layer is located between the asynchronous task layer and the synchronization task layer,and designed to ring buffer structure based on FIFO（First Input First Output）principle.Two semaphores are defined on the queue:free semaphores and data semaphores.The free semaphoresrecord the remaining space in the task queue,the initial value is equal to the queue length.The value of free semaphores pluses one when the synchronization task layer removes a task object from the queuing layer.The data semaphores records the number of tasks objects to be processed,the initial value is equal to 0.The value of data semaphores pluses one when the asynchronous task layer pushes the task object into the end of the queue.Each task object in the queue is associated with the data semaphore resources,effectively ensuring the data integrity,to eliminate the possibility of packet leakage.The synchronization layer performs high-level model business processing tasks（creating folders,file transfer/read and write file,terrain preprocessing,flow and runoff calculations,etc.）.The synchronization layer uses a dynamic thread pool model,consisting of a thread pool manager,a group of worker threads,and a predicted thread.The pool manager is responsible for thread scheduling.A free worker thread queue and a busy worker thread queue are designed to reduce the time complexity of the thread pool manager searching for idle work threads.When the task queue is not empty,the thread manager wakes up a worker thread from the free worker thread queue header and passes the task of the queue layer header to it,and then moves the thread to the busy worker thread queue.The worker thread returns to the free worker queue once the task is processed.This design can reduce the time complexity of the thread search from O（n）to O（1）.In order to avoid resource waste,the size of thread pool can be dynamically adjusted,prediction thread predict the number of threads needed in advance according to user request probability distribution,the pool manager creates/destroys thread according to predicted value.（5）BTOPMC/SCAU is used in Hanjiang river basin study.Hanjiang river basin is the second largest basin of Guangdong province.The basin area is 30112km².The elevation range is from 20 meters to 1500 meters.DEM,land cover,soil,satellite remote sensing and ground meteorological data are free downloaded from the Internet.The rainfall data of 193rainfall stations from1981 to1988 and the flow data of nine hydrological stations are collected.The data from 1981-1984 is used for parameter calibration,and the data from1985-1988 is used for model validation.The Hanjiang river basin is divided into nine sub-basins based on the location of nine hydrological stations and the natural boundaries of the basins.It includes 4 grade one sub-basin（GuanYinQiao,YangJiaFang,HeKoua and BaoKeng）,2 grade two sub-basin（ShuiKou and ShangKeng）,2 grade three sub-basin（XiKou and HengShan）and 1 grade four sub-basin（Chao’An）.According to the Hanjiang river basin’s natural terrain,the"top right to bottom left"is selected in the terrain module drainage basin selection options.The minimum elevation method is used to determine the direction of the flow of the grid.River Width Model is Zhong Xiangning Model.By default,the Voronoi diagram method is used to distribute the rainfall data.Flow calibration is calculated by Global Optimization of SCE-UA Parameters method.The convergence objective function uses geometric error（the geometric mean between Nash-Sutcliff efficiency and water balance errors on hydrological processes）.Soil Freezing and Thawing sub-model and Reservoir Operation sub-model are not used.All the above settings are completed by the system’s wizard menu and friendly user interface.The system provides an intuitive graphical visualization of the river basin simulation process and results.The results verify the rationality of the design scheme in this research.