Research On Comprehensive Evaluation Of Deep Learning Algorithms For Traffic Sign Recognition

Over the past few years,the development of artificial intelligence has led to the maturation of autonomous driving technology.Within this technology,traffic sign recognition is an essential component that provides information to vehicles for following traffic rules.Commonly used methods for traffic sign recognition include template matching,machine learning,and deep learning.Among these methods,deep learning is widely adopted due to its high precision and low labor requirements,and offers great potential for future development.However,existing evaluation indicators for deep learning models,such as accuracy rate,training loss rate,precision rate,recall rate,and F1-Score,primarily focus on recognition accuracy and are not suitable for evaluating traffic sign recognition under complex road conditions.Therefore,this study uses multiple evaluation indicators,including stickiness,time complexity,space complexity,and training speed,to comprehensively evaluate the results of traffic sign recognition from multiple perspectives.To conduct this research,we utilized the German Traffic Sign Dataset(GTSRB),the Belgian Traffic Sign Dataset(BTSC),and three classic image classification datasets(Cats & Dogs,Flower,and Car)as the test datasets.We also selected the convolutional neural network(CNN)as the primary test object,as it is frequently used for classification problems in deep learning models.By testing the various performance indicators of the classic neural networks Alexnet,GoogleNet,ResNet,and MobileNet,we analyzed the factors affecting the accuracy of traffic sign recognition and comprehensively evaluated the traffic sign recognition model.The main research work of this paper is as follows:(1)We tested the performance indicators of classical neural networks on traffic sign recognition datasets and other classic classification datasets.We evaluated five indicators representing recognition accuracy: accuracy rate,training loss rate,precision rate,recall rate,and F1-Score,and four indicators related to traffic sign recognition: robustness,training speed,time complexity,and space complexity.To eliminate dimensional differences between different features and avoid excessive influence of some performance indicators on the model evaluation results,we performed Min-Max Normalization on the obtained data.(2)We performed correlation analysis on the processed data to identify the performance indicators that have the greatest impact on the accuracy of traffic sign recognition.This paper mainly used the multiple linear regression equation to solve the correlation of multiple performance indicators and found that the training loss rate has the greatest impact on the training accuracy,while robustness has the least impact.Additionally,we found that robustness,time complexity,and training accuracy are negatively correlated.We calculated the Pearson Correlation Coefficient of the performance index and found that the training loss rate has the greatest impact on the accuracy rate,while the model parameter quantity has the least impact.Furthermore,the longer the training time and the greater the amount of floating-point operations,the higher the accuracy of the model.We concluded that robustness and accuracy cannot be combined.(3)We constructed an evaluation matrix from the performance indicators obtained in step 1,established a comprehensive evaluation model for traffic sign recognition,and conducted a comprehensive evaluation of the measured convolutional neural network.The weight of the comprehensive evaluation model was assigned through three methods: Subjective Weighting Method,Subjective Weighting Method,and Subjective Entropy Weighting Method.After comparing and adjusting parameters,we selected the most reasonable weight scheme and concluded that the overall score of the ResNet network model was the highest.Overall,this study provides a comprehensive evaluation of traffic sign recognition using multiple performance indicators,and identifies the key factors that affect its accuracy.