Design And Implementation Of EAST Experimental Data Gateway Based On RESTful And Microservice Architecture

EAST experimental data includes engineering data recording the operating status of engineering devices,diagnostic data recording discharge pulses,metadata describing diagnostic data,etc.Under the PB-level data scale,these experimental data are stored and retrieved through relational and non-relational databases.In a heterogeneous distributed environment,experimental data are scattered and stored in different forms and encoding formats,which exist independently of each other,which leads to the complexity of data acquisition.Based on this status quo,EAST,ITER,JT-60SA,KSTAR and other fusion experimental devices have developed and implemented their own UDA data access services for unified query and access to various experimental data.But for the UDA method of the monolithic architecture,as the demand increases,the boundaries of system functions are blurred,and there are hidden dangers in code editing.The program can only be expanded as a whole,and a unified technical solution is used,so it is not easy to introduce new frameworks and new technologies.At the same time,the UDA method generally needs to maintain clients of various programming languages,and the clients must go through a cumbersome compilation process under various operating system platforms to run,which also has the characteristics of insufficient usability and maintainability.Furthermore,on the basis of realizing the unified access function of fusion experiment data,related content of service governance and traffic management should also be added at the system level.Based on this background,this paper designs and implements the EAST experimental data gateway based on RESTful and microservice architecture.First of all,unlike the UDA method with high complexity,our method splits a single UDA into microservices that are independently deployed and run through different technical solutions according to the characteristics of the data source,so as to achieve decoupling at the business logic and technology stack levels.Secondly,compared to maintaining clients in various programming languages,this paper maps resources and operations to request methods and URIs of the HTTP protocol through the RESTful architecture,realizing cross-platform data access that is easier to use and maintain.Finally,on the basis of realizing the unified access function of fusion experiment data,this paper completes the related functions of flow control and service governance from the gateway level.Specifically,this paper designs and implements the EAST experimental data gateway based on the technical solution of the combination of Nginx and Flask back-end framework.On the one hand,the gateway provides a unified RESTful experimental data query service for various external terminal devices,enabling cross-platform data access without maintaining various programming language clients.On the other hand,the gateway internally accesses microservices implemented by different technology stacks to realize the decoupling of data access services.In addition,the gateway also serves as a centralized control point,which unifies the functions of protocol conversion and adaptation,service routing and governance,traffic management and fault tolerance,log recording and monitoring.In order to improve the maintainability of the system,this paper realizes the registration discovery and health detection of microservices through Consul.And based on Elastic APM,distributed link tracing is implemented to record related logs and display global topology relationships.The test results show that the EAST experimental data gateway meets the established requirements.