Analysis And Application Of Several Types Of Loss Functions In Deep Neural Networks

The loss function is the mathematical basis of deep neural network optimization problem.It influences model training by measuring the difference between the outputs and label values,and is a breakthrough in improving the efficiency of model training.Analyzing structures and variables of loss functions and explaining their mechanisms are challenges in the field of deep learning theory research.In this paper,we compare and analyze structure and role of functions based on the impact of loss functions on model training.And based on the existing deep neural network structure,we analyze the construction of the loss function and apply it to the field of digital image classification and object detection.The specific work and innovations are as follows:1.We try to study and analyze the back-propagation process of deep neural networks.To address the optimal solution problem of the loss function in neural network weight update,we analyze the information error back propagation methods of fully connected neural networks and convolutional neural networks.Based on the structure of the convolutional layer,pooling layer and loss function,a rigorous mathematical formula for parameter iteration is given one after another.In the experimental test phase,the accuracy of the weight update algorithm is verified by plotting the weight update change curve using the classical LeNet classification model.The detailed weight update calculation method helps to understand the signal transmission process in neural networks,and is an important guidance for the optimization of the network structure,the design of the loss function and the writing of the underlying source code.2.We try to improve the classification loss function based on intra-class and inter-class distances.To address the phenomenon that the Softmax loss function is not effective in increasing the inter-class distance and the problem of misclassification when dealing with multi-class classification problems,this paper proposes an improved classification loss function based on intra-class constraint and inter-class exclusion by adding an angular margin factor to control the size of the margin between classes and combining it with the Mahalanobis distance.We carry out experiments using the LeNet network to test the classification of ten categories from the MNIST dataset and the Fashion-MNIST dataset for validation test and to compare the effect with other existing loss functions.Compared with the margin learning function and the metric learning function,the function proposed can better improve intra-class compactness and inter-class separability,and the classification accuracy of the network is improved by 0.59%～2.43%.3.The classification loss function and the regression loss function are applied empirically.In the application of classification loss function,the theorem of CTC classification loss function is studied regarding the problem of inconsistency between output and true value length in recurrent neural network.We derived its forward-backward operator and calculate loss gradient,which are applied to natural scene text object detection algorithm to improve the accuracy of map coding recognition model.In the application of the regression loss function,we analyzed the definition and role of each parameter indicator of the EIoU regression loss function for the common target detection box regression problem.The EIoU regression loss function is applied to the constructed two-step modular lightweight detection network and the classification loss function based on intraclass constraints and inter-class exclusion is applied during image classification.The object detection accuracy of multi-source remote sensing images is improved combined with the reparameterization structure of the model.The F1 value is improved by 5%～10% compared with the YOLOv4 network and the Faster R-CNN based detection model.