On Nonsmooth Nonconvex Optimization Problems With Lagrange Neural Network

Many engineering applications such as signal processing,pattern recognition,optimal control can be abstracted as optimization problems,most of them are nonlinear optimization problem.Due to the fact that the traditional Lagrange neural network is not able to solve nonconvex optimization,by modifying the traditional Lagrange function,we propose two different neural networks to solve the nonlinear optimization problems that the objective function is nonsmooth nonconvex.Then theory analysis and numerical experiments are provided.Firstly,using the theory of penalty function,a new Lagrange neural network model is proposed based on the theory of the Lagrange multiplier.When the penalty factor tend to infinity,the optimization is easy to be a pathological problem,and the speed of convergence becomes slow.To overcome these drawbacks,we give a fixed parameter value and add a penalty term to the traditional Lagrange function.The optimization solution exist and the convergence trajectory of neural network tend to the set of critical points.Then two numerical experiments illustrate that the new model can solve the nonsmooth nonconvex optimization problem effectively.Secondly,adding a correlative item of equality constraint to the objective function,an augmented Lagrange neural network is proposed.The correlative item has no effect on obtaining the optimal solution and the parameter can make the objective function to be local convexification and accelerate the convergence process.The trajectory of the network model will reach the feasible region in finite time and finally tend to the set of equilibrium points.It is also proven that the critical points set contain the equilibrium points set.Finally the numerical experiments verify the effectiveness of the neural network model and the role of parameter is confirmed.