Research And Implementation Of Machine Learning Model Evaluation Technology

With the advent of the era of big data and the development of computer technology,artificial intelligence has been transformed fr-om relying on expert systems to algorithms.Among these algorithms,machine learning algorithms are important branches.Machine learning is applied to artificial intelligence software in the form of“model”.The quality of“model”determines the final effect of artificial intelligence application.Evaluation is an important means to ensure the quality of“model”and the quality of artificial intelligence applications.At present,model performance metrics have been used to evaluate model quality,but performance metrics have certain bottlenecks in terms of model stability,security,and specific business integration.The evaluation of machine learning models has not yet formed a complete index system.This paper focuses on"how to define the quality of machine learning model","how to build evaluation index system and evaluation model","how to perform evaluation".This paper constructs a machine learning model quality model,an index system,an evaluation model,and proposes relevant evaluation techniques and index processing techniques.This paper provides a comprehensive and reasonable index system and model safety evaluation method.This paper has certain research significance in ensuring the quality of machine learning models and the quality of artificial intelligence application.Firstly,this paper analyzes the difference and connection between machine learning model evaluation and software evaluation.Based on the in-depth analysis of the machine learning model characteristics,this paper extracted the six quality elements of machine learning,namely“performance”,“stability”,“model security”,“practicality”,“engineering efficiency”,“code security”,and establish a hierarchical initial index system.Then,through the literature analysis method,this paper summarize the principle of index construction,and propose the method of constructing the index system based on secondary screening.The first screening was done by qualitative analysis,and the second was the combination of expert questionnaire and qualitative analysis.Then,a simplified structure discriminant matrix algorithm is proposed,and the index weight is determined by the analytic hierarchy process.The index system of the handwritten digit recognition model is established by the method proposed in this paper.Secondly,this paper proposes a formal general flow of model mathematical attribute evaluation.Then evaluates the performance of the handwritten digit recognition model and the software defect prediction model,and the effectiveness and limitations of the performance metrics are analyzed.Then,the first kind of robustness metric robustness₁ is proposed and the effectiveness of robustness₁ is verified experimentally.Then,based on the in-depth analysis of the principle of anti-sample construction,the second type of robustness measure Defense is defined,which is the model's defense ability against attack.A second type of robustness evaluation method based on adversarial attacks,and verify the validity of the methods and indicators.Next,calculation formula for each index processing is proposed.Finally,on the basis of theoretical research,the design and implementation of the image classification model evaluation system is carried out,and the system verification and function display are carried out.This paper focuses on the machine learning model evaluation index system and the evaluation technology of model mathematical attributes.The proposed first type of robustness metrics can distinguish the model's ability to deal with rational anomaly data.The proposed second type of robustness metric can distinguish the model's resistance to against sample attacks.