Research On Deep Learning Algorithms Based On Nonlinear Dictionary Learning

In recent years,artificial intelligence technology has developed rapidly and has been widely used in various industries around the world to promote global economic development.The dictionary learning method in machine learning,which is capable of downscaling and sparse representation of huge data sets,has been widely used.In view of the difficulty of linear dictionary learning to meet the requirements for nonlinear signal representation and the fact that the current methods for training neural networks are mainly gradient descent methods and their variants,while gradient-based methods have weaknesses such as gradient vanishing,easy fall into local minima,and slow convergence according to optimization theory,the following research is carried out in this thesis.1.In this thesis,a novel nonlinear dictionary learning model is proposed to meet the requirements for nonlinear signal processing.In the sparse coding stage,based on the orthogonal matching pursuit(OMP)algorithm,we design the nonlinear orthogonal matching pursuit(NL-OMP)algorithm to update the coefficient matrix.In the dictionary updating stage,based on the method of optimal directions(MOD)and K-singular value decomposition(K-SVD)algorithms,we design the nonlinear optimal directions method(NL-MOD)and the nonlinear K-singular value decomposition(NL-KSVD)algorithms to update the dictionary,respectively.Finally,the effectiveness of the proposed nonlinear dictionary learning method is verified by means of numerical experiments.2.A method based on non-gradient training feedforward neural networks is proposed,i.e.,a reconstruction of the non-gradient training feedforward neural network method using a nonlinear dictionary learning model.In order to implement the constraint more precisely,this thesis adds a Lagrange multiplier term to the objective function to obtain higher accuracy.The objective function is approximated by a boosted proximity operator method to relax it into a multi-block convex optimization problem.The method is a multi-block convex optimization problem at all levels of the neural network with weights,activations,and Lagrange multiplier terms,which allows the use of the block coordinate descent(BCD)to update the weights and activations at each layer of the neural network.This method uses only the activation function itself and not its derivatives,which avoids the use of gradientbased methods and thus overcomes the drawbacks associated with the use of gradient methods.3.To further validate the effectiveness of the proposed non-gradient-training neural network algorithm,extensive experiments are conducted in this thesis.First,experiments are conducted using publicly available datasets,and the convergence performance is compared with the gradient back-propagation-based method.The results show that the proposed algorithm has better convergence performance.Second,experiments are conducted using a variety of different neural network structures to verify that the algorithm can adapt to different network structures and obtain better performance.Finally,experiments are conducted by replacing a variety of nonlinear activation functions to effectively verify the advantages of the proposed algorithm.These experimental results show that the proposed algorithm for constructing feedforward neural networks based on nonlinear dictionary learning models for non-gradient training is effective.