Research And Implementation On Key Technology Of AI Capacity Servitization

In recent years,artificial intelligence technology has made considerable breakthroughs in various fields,but the process of implementing artificial intelligence technology is not optimistic.On one hand,each AI system is independent,and there has been a lot of repetitive development work.On the other hand,in the process of AI algorithm research and development for actual business,many technical debts have been accumulated in data governance,development,deployment,operation and maintenance management,which hinders the process of algorithm research and development.In order to solve the above pain points,AI middle platform and MLOps technologies have emerged to complete the reuse of AI business-related components and automate the management of the entire life cycle of AI algorithms.At present,many excellent MLOps frameworks have emerged.However,these frameworks mainly focus on building automated pipelines for AI algorithms from training to deployment and dealing with data drift.For AI-oriented businesses,especially data governance related to deep learning,data set governance,feature governance,data visualization and other issues still lack better solutions.In this thesis,aiming at solving the problems of repeated development in the process of AI service construction,an AI platform is designed and implemented using the microservice architecture,and the common components,including data processing,algorithm training,algorithm testing,algorithm deployment and other common components in different AI service development processes are integrated into the middle platform,the entire platform is divided into five subsystems:Container Cloud Platform,Data Platform,Training Center,Test Center,and Deployment Center,an algorithm library is also designed to accumulate the algorithms in the platform.In addition,supporting services such as GPU scheduling,cluster resource scheduler,monitoring system,and database are also developed.Aiming at the pain points in the process of data governance and visualization in the AI business development,this thesis analyzes and sorts out the characteristics of data processing in AI business and the requirements that the AI platform needs to meet in order to support these data processing processes.Based on these requirements and features,this thesis proposes key concepts such as Datasets,Dataset Views,Dataset Attributes,Features,Data Pipelines,and Visualization Units.Based on these concepts,an AI business-oriented data platform is designed and implemented.In the designed data platform,users can create their own datasets,manage,visualize and analyze the features and data files in the datasets,create their own data processing components conveniently,and flexibly reuse data processing components,and the development processes of data components is more convenient than other MLOps frameworks.However,while the platform has convenient data governance capabilities,the cost of algorithm integrating into the platform is also increasing,causing decreasing usability and maintainability of the platform,which is because the diversity of algorithms,the multi-source and heterogeneous characteristics of data processed in AI business constitute a contradiction with the unity of algorithm integration process.In response to deal with this issue,this thesis explores the algorithm integration process in the platform,compares the centralized algorithm code library based on the unified architecture and the discrete algorithm project library based on the SDK,and innovatively extracting the concept of data access point in algorithm and proposing the data structure of DataAccesser and designing related SDK to accelerate the integration process,so that the algorithm can conveniently complete the data access while using the data governance function of the platform,which simplifies the integration process of the algorithm in the platform.The designed platform makes the algorithms easy to integrate and develop,can complete training,testing,and deployment with one click,can conveniently increase,decrease,and switch data sets during the training and testing process,and make the data used in these processes automatically documented,reducing the technical debt when developing algorithms.It has great practical value in scenarios such as incremental learning that require frequent increases or decreases of data for training or testing.And the overall use process of the platform is all paged,which has good usability.Finally,this thesis takes the video violence recognition application as an example,integrates the SlowFast algorithm into the platform and uses this algorithm to model the video violence recognition application.The perspective of data and information is used for optimizing the inference performance of the algorithm in violent scenes.Finally,the thesis shows how to use the designed platform to quickly iterate data for AI algorithms and finally deploy it as a service.