Research On Parallel Convolutional Neural Network And Catastrophic Forget Problem

In recent years,neural networks have evoked waves of research in both academia and industry.This boom has driven deep learning technology to achieve stunning results in many fields such as computer vision,natural language processing,and man-machine confrontation.result.In a multi-tasking scenario,the neural network can achieve near-perfect results in a single task,but when switching to a new task,the neural network will change the previous parameters on the old task during the training of the new task.The rewriting of this parameter will reduce the performance of the new network on the old task.In the field of cognitive science,these phenomena are called catastrophic forgetting of neural networks,and catastrophic forgetting is also considered to be one of the many limitations of neural networks.To solve this problem,this paper proposes a dual-branch parallel convolutional neural network architecture based on the traditional convolutional neural network structure.The entire architecture has two left and right branch structures.The old and new tasks respectively enter the left and right different parallel branches,and the lower branches separately perform calculations.The two branches merge into one backbone in the high-level part of the parallel architecture for classification.The entire architecture uses neural networks to automatically characterize features and relate the intrinsic features of the old and new tasks to some extent to ease the catastrophic forgetting problem.Further,this paper adds a threshold unit control module based on the parallel convolutional neural network architecture.The output value of the module will eventually be applied to the total loss function of the parallel architecture.The input of the threshold module is the convolution in the task branch.The output of the layer,the output of the threshold module is a threshold between 0 and 1,and the output threshold can be used to more effectively control the effect of the left and right task branches in the parallel architecture on the entire model.In the loss function design,the two-branch parallel convolutional neural network calculates its own losses on the left and right branches respectively,and obtains the entire parallel convolutional nerve by weighted summing of the thresholds calculated by the threshold module and the losses corresponding to the left and right branches.The total loss of the network,the goal of the entire parallel architecture training is to optimize the total loss.Experiments show that parallel convolutional neural network can effectively alleviate the catastrophic forgetting problem of neural networks.In addition,during the research process,we found that the Dropout method also has a certain effect on alleviating the catastrophic forgetting of neural networks.