Research On Loss Function Of Convolutional Neural Network For Image Classification

Image classification is one of the four basic tasks of computer vision,and it is also the basis of other three computer vision tasks.In order to obtain better image classifi-cation performance,the core problem is how to obtain better and more abstract feature representation.As one kind of representation learning methods with multiple levels of representation,convolutional neural network(CNN)can automatically extracts feature representation.Since these feature representations are superior to traditional algorithm-s,CNN-based algorithms have become the dominant technique for image classification task.As an indispensable component of CNN,loss function determines the updating pro-cess of the whole parameters in CNN,and thus determines the feature representation of the raw image to some extent.Therefore,research on loss function has become one of the hottest research topics in improving the generalization performance of CNN model-s.However,the existing works on loss function,especially for image classification,still have the following deficiencies:1)The existing work still focuses on single loss function design and lacks the discussion and analysis of utilizing multi-loss function.2)The ex-isting work mainly focusing on designing new loss function analyzing the characteristics of feature representation in the penultimate layer of CNNs,and lacks the exploration and analysis of the characteristics of the feature representation in other layers of CNNs,which results in limitations in theoretical analysis.3)The existing work is mainly focus on face recognition,image retrieval,ReID and other tasks.The effect is limited when applying the methods from these works in image classification task,as they have different decision rules during testing phase.Addressing these problems,this dissertation proposes a nov-el training framework,Loss Transferring,and the basic guides when utilizing multiple loss functions to train CNN models,which results in better classification performance for CNNs.At the same time,this dissertation also focuses on analyzing the characteristics of feature representation in the last layer and decision layer of CNNs,and then propose two new properties for CNNs,i.e.,Location Property and Unbiased Property.Based on these two new properties,this thesis also design different loss functions to improve the classification performance of CNN models.More specifically,the main contributions of this dissertation are in the following three folds:1)To utilize multiple loss functions to train CNN models,a novel training frame-work,Loss Transferring(LT),is proposed.LT adopts the idea of transfer learning to loss function domain,which enables CNN models to combine the knowledge of object learned from different loss functions by transferring the knowledge of objects learned via one loss function to another.Meanwhile,In order to solve the problem of choosing different loss functions in different training stages.According to these two guides,a new loss function,Near Classifier Hyper-Plane(N-CHP)loss,is further introduced in the last layer of CNN models to build two specific training methods,LTMSE,softmax and LTN-CHP,sofmax,with softmax and MSE.Comprehensive experimental results on four benchmark datasets and different CNN models indicate the superiority of LT and the effectiveness of the proposed two basic guides.2)By analyzing the characteristic of feature representation in the last layer,a nov-el property "location property" of CNN is proposed,which indicates that the key of CNN-based methods for image classification is to find the optimal feature embedding location in the last layer.In order to find the optimal location,two feature embedding directions,the PE-direction and SE-direction,are proposed as the theoretical guidance.And meanwhile,the optimality of S-OFP in the SE-direction is also theoretically proved.In addition,a novel loss-based optimization framework,LP-loss,is proposed to make fea-ture representation move in the PE-direction and SE-direction simultaneously,in which LP-loss contains two losses,LPPE and LPSE.Comprehensive experimental results on four benchmark datasets and different CNN models verify the correctness of location property and indicate the effectiveness of LP-loss.3)By analyzing the characteristic of feature representation in the decision layer,a novel property "unbiased property" of CNN is proposed,which indicates that elements of feature representation in decision layer corresponding to error categories should be e-qual to each other.In order to achieve unbiased property,two novel loss function,Minmax Probability Constraint(MMPC)loss and e-MMPC loss,are proposed.These two loss-es enable CNN models to maximize correct category probability and minimize all error categories probability simultaneously during training phase.To illustrate the effective-ness of MMPC and e-MMPC theoretically,analysis from a perspective of gradient shows that both two losses can alleviate the vanishing gradient problem.To further improve the performance of MMPC or e-MMCP,a two-phase relearning strategy is proposed.Com-prehensive experimental results on four benchmark datasets and different CNN models verify the correctness of unbiased property and indicate the effectiveness of MMPC and e-MMPC.In summary,aiming to address multiple-loss training strategy problem,this disserta-tion proposes loss transferring(LT),which can combine different knowledge learned from different loss function to improve the generalization ability of CNN models.Meanwhile,by analyzing the characteristics of feature representation in last layer and decision layer,location property and unbiased property are proposed.In order to achieve these two prop-erties,a novel loss-based optimization framework,LP-loss and MMPC/e-MMPC loss are proposed.Both two methods can improve the classification performance.