| Image recognition is the most fundamental task in the field of computer vision.The image recognition method with Deep Convolutional Neural Network(DCNN)is the principal method,achieving the state-of-the-art performance.The methods based on DCNN rely on the static environment,and training data with huge data scale and a balanced class distribution.In the complicated real world,the data distribution is often imbalanced,the volume of data is continually increasing,and new classes are constantly emerging.These factors bring great challenge for image recognition.When dealing with data imbalance,the DCNN model obtained by the general training algorithm is prone to the majority classes,and often overfits on the minority classes,having a poor performance.Data imbalance is usually coupled with data complexity.Data complexity means the data have high inter-class similarity,and large intraclass contrast,affecting the performance further.Faced with the issue of data increase,owing to the learning of new knowledge and data imbalance between old and new classes,the performance of incremental learning model on the old data usually declines,which is called catastrophic forgetting.From the view of rectifying and constraining the representation learning,the dissertation studies these challenges based on the Deep Convolutional Neural Network,and proposes the corresponding image recognition methods.The main contributions can be summarized as follows:(1)To alleviate the influence of data imbalance and data complexity,we design an image recognition method using Covariance Discriminant loss(Cov D Loss).We employ the Cov D loss considering the first and second order distance to provide more constraints for the representation learning of the hard samples.Therefore,the deep features of different classes are separated effectively.Further,we formulate the corresponding hard sample mining algorithm,to select the misclassified samples or samples with improper feature representation.The experiments on the ISIC 2018 Skin Lesion Analysis dataset and imbalanced FGVC-Aircrafts dataset verify the designed method rectifies the deep features of minority classes effectively,and performs well on the task of imbalanced data classification.(2)Considering internal structure of data,we design an image recognition method with Cluster-based metric loss,to deal with the data imbalance and data complexity.Specifically,we use k-means method to mine the clustering structure,and employ the cross-entropy loss and Cluster-based metric loss to perform joint optimization,boosting the representation learning and classification ability.The designed Cluster-based metric loss is utilized to constrain the relationship between clustering distance,intra-class distance,and inter-class distance,rectifying the deep features,and ensuring the margins between classes.The comparison experiments on the imbalanced Stanford Dogs dataset and Describable Textures dataset demonstrate our method consistently outperforms the other methods.And by the ablation experiments,we have verified the effectiveness of each term in the proposed loss.(3)In terms of catastrophic forgetting caused by the learning of new knowledge and data imbalance between old and new classes,we embed the model-based method into the memorybased method,and propose a novel Exemplar-Supported Representation for class Incremental Learning(ESRIL)method.First,we use Memory Aware Synapses to estimate the parameter importance of the old model,alleviating the weight drift,and preserving the feature separability.Compared to the other incremental learning methods,as we resize the image first,DCNN pretrained on Image Net can be adopted as the backbone in our MAS module,reducing the training epochs and alleviating the changes of model weights.Second,scalable Exemplar-based Subspace Clustering(ESC)is utilized to select and rank exemplars,guaranteeing the sufficiency and diversity of exemplars.Third,the Nearest Class Multiple Centroids(NCMC)classifier is used as the classifier to replace the fully connected layer of MAS,reducing the training epochs,and mitigating the impact of data imbalance of old and new data when the criterion is met.The comparison experiments and ablation experiments show that the proposed method has an excellent performance,and outperforms the comparison methods.(4)We propose a class-incremental learning method with Dual-branch Deep Nearest Class Mean classifier(Dual DNCM).In the ESRIL method,the modules of feature extraction and classifier are obtained separately,have a relatively low matching degree.We present a method with joint training,Dual DNCM,to reduce the catastrophic forgetting further.First,the proposed method employs an ensemble learning way to train original and re-balanced branches,reducing the impact of data imbalance of the old and new classes.Second,the activation transfer loss is utilized to reduce the catastrophic forgetting from the view of feature representation,preserving the feature separability on old classes.Third,we use the Nearest Class Mean classifier with natural advantages as the classifier.Moreover,we formulate a joint training algorithm for the feature extraction and classifier,to boost module matching degree.Comparison experiments indicate our proposed method perform well on the old and new classes.And the ablation study shows the importance and necessity of dual-branch structure,joint training,and activation transfer loss. |
