Research On Deep Convolutional Neural Network Based On Sample Distribution

With the rapid development of the information age,as a very powerful way to describe the object,images have gradually become an important means for people to obtain information,analyze information,and use information.In recent years,deep learning has made great progress in the field of computer vision,achieved the best results in many fields,and brought new opportunities for the development of multiple fields,such as image segmentation,image classification,and object detection.It promotes the development of various commercial applications.As one of the typical representatives,image classification algorithms based on deep learning applied in areas such as autopilot,intelligent monitoring,etc.,have gradually become an important direction in scientific research.In the training process of deep neural network,practitioners can often get a good result by adjusting the parameters several times.However,we,especially the beginners,often encounter problems when using deep convolutional neural networks.Optimization of deep learning is no longer an imminent problem,due to various gradient descent methods and the improvements of network structure,including activation functions,the connectivity style,etc.Then the actual application depends on the generalization ability,which determines whether a network is effective.Regularization is an efficient way to improve the generalization ability of deep CNN,because it makes it possible to train more complex models while maintaining a lower overfitting.In this paper,we propose to optimize the feature boundary of deep CNN through a two-stage training method(pre-training process and implicit regularization training process)to reduce the overfitting problem.In the pre-training stage,we train a network model to extract the image representation for anomaly detection.In the implicit regularization training stage,we re-train the network based on the anomaly detection results to regularize the feature boundary and make it converge in the proper position.Experimental results on five image classification benchmarks show that the two-stage training method achieves a state-of-the-art performance and that it,in conjunction with more complicated anomaly detection algorithm,obtains better results.Finally,we use a variety of strategies to explore and analyze how implicit regularization plays a role in the two-stage training process.Furthermore,we explain how implicit regularization can be interpreted as data augmentation and model ensemble.The innovation of this article mainly includes the following three points.Innovation point one:by analyzing the distribution characteristics of training samples in source domain,we propose an anomaly detection method based on sample density,which lays the foundation for the implicit regularization training stage of network.Innovation point two:we have established a new regularization method in the second training stages of the network to reduce the over-fitting problem of the neural network.Innovation point three:we prove the influence of the two-stage training method on the neural network training process from the theory to the experiments by visualization technique.The experimental results show that even with some more advanced networks,our two-stage training method still has the advantage.In fact,the training strategy proposed in this article is not limited to this.In future work,we plan to apply this training strategy to a more advanced depth network structure.