Research On Gaussian Distribution-based Orthogonal Decomposition Loss For Classification

In recent years,deep neural networks have achieved great success and are now widely used in various fields of life,such as image recognition,face recognition,and speech recognition.With the continuous optimization of network structure and loss functions,deep neural networks have significantly improved the performance of various complex classification tasks.The loss function is an indispensable part of deep learning,and there are various loss functions for different tasks,such as MSE,BCE,etc.There are many studies on the performance of the loss function.A good loss function should theoretically make the distribution of features of the same class in the datasets more compact,and the distribution of features of different class is more separated.This paper presents a novel loss function,namely,GO loss,for classification.Most existing methods,such as center loss and contrastive loss,dynamically determine the convergence direction of the sample features during the training process.By contrast,GO loss decomposes the convergence direction into two mutually orthogonal components,namely,tangential and radial directions,and conducts optimization on them separately.The two components theoretically affect the inter-class separation and the intra-class compactness of the distribution of the sample features,respectively.Thus,separately minimizing losses on them can avoid the effects of their optimization.Accordingly,a stable convergence center can be obtained for each of them.Moreover,This paper assumes that the two components follow Gaussian distribution,which is proven as an effective manner to accurately model training features for improving the classification effects.Experiments on multiple classification benchmarks,such as MNIST,CIFAR,and Image Net,demonstrate the effectiveness of GO loss.This paper mainly studies from the following aspects:(1)This paper proposes a new optimization perspective.Specifically,This paper considers the optimization problem from the perspective of convergence direction.(2)This paper decomposes the convergence direction into two mutually orthogonal components,namely,tangential and radial directions,and conduct optimization on them separately.(3)This paper decouples the direction and norm of feature to avoid their interference with each other during the optimization process.(4)This paper uses the direction and norm of feature to associate with the inter-class separability and intraclass compactness,respectively.(5)This paper uses Gaussian distribution to guide the optimization processes on direction and norm of feature.