The Design And Implementation Of Sophon Cloud Model Deployment System

As big data technology is widespread,people's demand for the complicated Artificial Intelligence(AI)model is growing.Being a cornerstone of AI development,Machine Learning(ML)plays a significant role in science research and production practice.For example,lots of enterprises utilize ML technologies such as video analysis,object detection and speech recognition as parts of their real-time flow analysis or deep batch analysis pipelines,in order to provide products that are more efficient,performance-oriented and individualization in terms of consumers' preference.Deploying ML model on a large scale in an extensible distributed environment is a crucial step for enterprises implement ML model to product circumstance.However,as a complex task,deployment of ML model that requires professional skills and trial and error may be faced with several issues: rapid and flexible deployment of ML model,heterogeneity of hardware,provision of low latency and large throughput interface services,etc.Therefore,enterprises need a reliable platform of ML model deployment,arrangement and management where they are able to speed up the timeline of their services.Further,the platform enables enterprises to decrease the cost for human captain and operation generated from large scale ML model in the production environment,which means they could effectively manage the life cycle of ML work flow at the lowest cost.This thesis provides a comprehensive solution to aforementioned problems in terms of the designation and implementation of the Sophon Cloud Model Deployment System.1.This thesis puts forward the research and develop model deployment system to shorten cycle for enterprise applying ML model,by analyzing the practical significance of AI application and the current issues for enterprises.2.Based on the analysis of the current situation and shortcomings of the model deployment system technology in both domestic and oversea fields,this thesis summarizes the full requirements of the model deployment system.In addition to the summary,this thesis also points out the overall structural design of the system and elaborates designs of five functional module: the cluster management module,the model shelf management module,the model online management module,the model performance monitoring management module and the approval management module.3.Depending on the framework of Spring Cloud,this thesis presents implementation mechanism of each functional modules and testifies the realizability of the system through the actual online operation and system test following through the detailed design of the Sophon Cloud system.Since the Sophon Cloud Model Deployment System was online,the system has been running stably,which enables users at enterprise level to deploy ML model more quickly and flexibly.In addition,this system accelerates the landing speed of user AI products,reduces the cost of users deploying ML model,and provides users with model prediction services of large throughput and low latency.