Research On Optimization And Application Technology Of Gradient Descent Algorithm In Deep Learning

In recent years,as the core driving force of a new round of industrial reform,artificial intelligence represented by deep learning has developed rapidly,benefiting all walks of life.As one of the three key elements,optimization algorithm further promotes the extensive use and development of deep learning in practical applications.As the mainstream optimization algorithm,the stochastic gradient descent algorithm with momentum updates the weight based on the current gradient and historical gradient.However,in the actual training process,oscillation(overshoot)may be encountered,which may delay the convergence,increase the training cost,and reduce the model performance.Taking epidemic prevention and control as an example,since the outbreak of the novel coronavirus,it has gradually developed into a major public health event worldwide,which has had a devastating impact on the health and well-being of the global population.Since the detection ability of the novel coronavirus lags far behind the spread speed of the virus,it does not match the strong infection ability of the virus.Recently,the global increase in confirmed cases is more than one million per day,and the medical system is under enormous pressure.Whether artificial intelligence can be used to improve the speed and efficiency of medical diagnosis has become a hot research topic in the application of artificial intelligence in the industry.Through the analysis of the research status at home and abroad,it can be concluded that there are still the following problems in the existing research work:1)How to effectively guide the network model training process through the optimization algorithm,achieve the convergence state faster and more accurately,complete the model optimization,and promote the rapid deployment and early application of the model to promte the rapid solution and practical application of the novel coronavirus detection problem in this scenario.On the premise of ensuring the accuracy,the time-consuming of the whole optimization process is effectively reduced and the deployment is carried out as soon as possible,which greatly improves the efficiency of novel coronavirus detection,effectively curbs the spread of diseases and responds to the spread of epidemics.2)How to solve the problem that SGD-M algorithm encounters oscillation and delays convergence in the process of optimization,it is necessary to systematically analyze the influence of current gradient information and historical gradient information on the update of weight parameters,reveal the causes of convergence and oscillation,and construct an algorithm model based on gradient information,focusing on its proportional term and integral term.By improving the lag and oscillation of weight updating,an algorithm updating strategy based on the comprehensive trade-off of proportion,integration and differentiation is formed,which provides a certain degree of advance for the optimization process and accelerates the optimization.In view of the above technical and scientific problems,we first draw on the relevant theories in the control field,interpret the entire optimization process from the perspective of dynamic system,and analyze the influencing factors of weight updating strategy.In this paper,two improved optimization algorithms are innovatively proposed.By comparing with the related experiments of various optimization algorithms,the average acceleration effect of about 20 % can be achieved under the premise of ensuring high precision and not reducing the model performance,and the convergence speed is faster,which proves the effectiveness and good generalization ability of the designed optimization algorithm.Secondly,in order to promote the rapid deployment of artificial intelligence-assisted new coronavirus detection in practical application scenarios and solve the deficiency and inefficiency of detection ability,the optimization algorithm in this paper is used to complete the rapid training and optimization of the model,reduce the time-consuming of the optimization process,and realize the rapid and accurate detection function of the novel coronavirus on the web side by deployment,and assist the decision-making of doctors to improve the detection efficiency,which is of great significance to the prevention and control of the epidemic.The main work accomplished is as follows:1)The optimization process of deep neural network is analyzed from the perspective of dynamic system,and the similarity between deep neural network and feedback-based closed-loop control system is analyzed.In order to improve the weight updating strategy of SGD-M,a PID algorithm with saturation function constraint is proposed to slow down the oscillation and accelerate the convergence.Different from previous studies,the saturation function constraint for differential term(D)is designed for the first time by giving it an adaptive amplitude,which depends on the absolute value of proportion(P)and integral(I),and effectively weighs the contribution and role of the three in the update process.Several mainstream network models are used to carry out experiments on different datasets.Compared with other optimization algorithms,the average training convergence speed is increased by 20 %,and the maximum is up to32 %.It shows that it can alleviate the oscillation to accelerate the optimization process and ensure high precision.2)A hybrid algorithm is proposed based on the integral separation theory with high computational complexity and hyper-parameters brought by the high-order dynamic system optimization algorithm.Based on Lyapunov analysis of SGD and SGD-M algorithm,the algorithm is designed as a hybrid switched dynamic system to effectively deal with various stages of deep neural network optimization process.Firstly,the class proportion-integral form is used to ensure high precision,and then the algorithm is improved based on the viewpoint of hybrid dynamic system as a hybrid form of oscillation suppression to accelerate optimization.In addition,through the theoretical analysis and elaboration of the algorithm,it is the first time to explore the super parameter selection conditions that maintain the stability of training and complete the guidance of super parameter selection.Several mainstream network models are used to carry out experiments on different datasets.Compared with other optimization algorithms,the average training convergence speed is increased by 25 %,and the maximum is increased by 42 %.It further shows the superior performance of the designed optimization algorithm and promotes the application of the hybrid optimization algorithm.3)At present,the global problem of the prevalence of new coronavirus continues to deteriorate,and the increase in the number of infections and deaths per day has not slowed down.As an important part of epidemic prevention and control,the medical system is faced with huge COVID-19 detection pressure every day.Combined with the proposed optimization algorithm,the COVID-Net network model is used to train and optimize the model on the COVID-19 dedicated benchmark data set.Based on the lightweight Web application framework Flask,Gunicorn server and Open CV library,a rapid detection function of COVID-19 on the web side is realized.In addition,the effectiveness and universality of the proposed optimization algorithm are further demonstrated.