Research On Image Restoration Method Based On Memristive Convolution Neural Network

With the rapid development of artificial intelligence technology,computer vision has been widely used in various fields of people’s production and life,such as:intelligent traffic monitoring,military reconnaissance,automatic driving,medical and health care,etc.Computer vision systems require clear and clean image data as input.However,in the face of common weather environments such as rain,haze and poor lighting,image data captured by outdoor cameras and surveillance equipment severely damage visual effect,resulting in problems such as background blur,detail loss,and insufficient brightness.These degraded images greatly degrade the performance of computer vision systems.Therefore,it is of great significance and practical application value to study the image restoration method so that the machine can "see" clearly.Traditional image restoration methods rely on pre-determined prior knowledge and constraints to build and optimize the model.These priori-based methods typically perform restoration operations in other clean regions,resulting in smooth transitions in these regions.In addition,the limited generalization capability of these predefined models leads to poor detail recovery of the restoration results.At the same time,due to a large number of parameters to be optimized,traditional image restoration algorithms are often very time consuming.With the rapid development of deep neural network technology,image restoration methods that rely on deep learning are gradually replacing traditional methods.Deep learning methods complete the task of image restoration by establishing a complex mapping between the degraded image and the real background through neural networks.Compared to traditional image restoration methods,deep learning methods rely on powerful feature extraction capabilities to achieve better image restoration effects.However,most of these methods are designed for a specific task and they have huge number of parameters and computations,which is not conducive to hardware implementation,making it difficult to deploy them to the end-side intelligent system with limited computing resources.Recently,an increasing number of scholars have investigated the application of memristors to hardware implementations of deep neural networks due to their various desirable features such as nanoscale size,non-volatile storage,programmability,and CMOS compatibility.However,most of the existing neural network hardware implementation circuits are designed based on generic frameworks,and the research of multi-task image restoration system based on memristive neuromorphic computing architecture is not perfect.To address the above issues,this thesis combines memristors and convolutional neural networks,studies the image restoration method based on convolutional neural network,and uses the memristor as the basic computing device to construct the full circuit implementation scheme of the network,and the main innovative work is summarized as follows:(1)In view of the fact that most existing image rain removal methods cannot achieve a good balance between removing rain streaks and restoring the corresponding background details,this thesis proposes an effective dual path convolutional network(DPCN)for single image rain removal.DPCN can locate,extract and separate the rain streaks by using multiple dual path units.Firstly,considering the irregularity of the size,density and shape distribution of rain streaks,a pixel-wise attention mechanism is applied to pinpoint the position of rain streaks.Simultaneously,for these rain streaks distributed in different regions,a multi-scale aggregation method is proposed to extract and fuse the features of different scales.In addition,for some backgrounds whose texture details are similar to rain streaks,a self-calibration operation is introduced to separate rain streaks from these background details by adaptively constructing long-range spatial and internal channel dependencies at each spatial location.By cleverly combining multiple dual path units through a dual path topology,DPCN obtains rain removal results that aresssss closer to the real background,and remove the rain streaks to a large extents.Quantitative and qualitative experimental results on both synthetic and real datasets show that DPCN can effectively remove various types of rain streaks while recovering the texture details of the real background.(2)To address the problem that most existing image restoration methods are only suitable for a single task and not conductive to end-side deployment,this thesis proposes an image restoration method based on global self-attention memristive convolutional neural network(GSA-MCNN),and gives a circuit design and implementation scheme for the GSA-MCNN based on memristors.The GSA-MCNN is capable of extracting both global and local information from images,and has full convolutional properties,which makes it flexible to be applied to images of different resolutions.Firstly,two global attention modules are designed to complete the global relationships modeling and reasoning: the global spatial attention module is used to model the global spatial relationships between the pixels of the feature maps,the global channel attention module is used to explore the global relationships across channels.In addition,a multi-scale local information extraction module is designed to handle image regions with complex texture details.Finally,a full-circuit implementation scheme of these three modules is provided,using a modular design to complete the circuit design of the entire GSA-MCNN.Benefiting from the programmability of the memristor crossbar,three kinds of image restoration tasks:image deraining,low-light image enhancement and image dehazing are realized on the same circuit template by tuning the configuration parameters.Experimental comparisons with over twenty state-of-the-art methods on ten public datasets show that the proposed GSA-MCNN has superiority in color reproduction and texture detail fidelity.