An Improved BP Wavelet Network And Its Applications

Wavelet Networks (WN) is the combination of Wavelet Analysistheory and Artificial Neural Networks theory (ANN). It not onlypossesses the wavelet transform's time-frequency localizationability, but also makes full use of the property of self-learning ofANN, which enables WN to have better ability of approximation andbetter fault-tolerance capacity. Because of its particular merits,WN is widely used in many fields, such as signal processing, functionapproximation, data forecast, system identification and patternrecognition.Back-Propagation (BP) algorithm is the typic algorithm forfeed-forward neural networks, which is simple and effective. But thefundament of BP algorithm inevitably enables it to have lowconvergence rate, be sensitive to initial network parameters and easyto fall into local minimum. In order to avoid these problems in thelearning process, many scholars proposed many improved algorithms,such as the momentum method, the stochastic gradient method, theconjugate gradient method, BFGS, least square method, sector methodand so on. Many algorithms have high convergence rate, but they areeasy to fall into local minimum. In view of this, some scholarsintroduced some global convergence algorithms like Genetic Algorithms,Simulate Anneal Arithmetic. However, because of the complexity anddifficulties of these algorithms and their applications, relevantstudies and applications are few. This thesis paper introduces a newalgorithm--DY ? HS to WN. The improved BP wavelet networks based onDY ? HS has higher convergence rate and it has global convergenceeven if the error function is in weak condition.The DY ? HS algorithm is a special kind of conjugate gradientmethod. It has the general form of conjugate gradient method:xk +1 = xk + αk dk1kkk k kd = ??? ?? gg +β d? 若若kk ≥=12α k is step factor, g k is the gradient vector of E at xk , andβ k is parameter,β k = max {0 ,min { β kH S ,βkDY}}11 1THS k kk Tk kg yβ = d ? y??21 1DYkk Tk kgβ = d ? y?y k ?1 = g k ? gk?1Compared with other algorithms, DY ? HS has higher convergencerate, and can guarantee the global convergence just with non-preciseWolfe line search. So the BP WN based on this algorithm has betterlearning ability.Here is the modeling process of this improved BP WN:Step 1: design WN architecture according to actual problem.Step 2: initialize network parameters. The weight coefficientW could be initialized with the random numbers in (0,1) whiletranslation factor b could be initialized with the random numbers in( ? μ , μ ), μ> 0. Scale factor a has more requirements because it mustinsure that it should have the capacities of dilation and compression.Then we use x1 to denote the initial parameter vector, andinitialize ε > 0 and e > 0.Step 3: feed-forward computation. Put all the samples (t k , T k)( k = 1,2, , K) into the network, then compute the error function E .IfE < e, stop, x1 is the optimal parameter vector.Else go to Step 4.Step 4: feed-back revision. First we should initialize the maximumof iterative times K , then compute β k according to DY ? HSalgorithm, and compute α k according to non-precise Wolfe linesearch.Gradually modify network parameter vector until E < e or g k≤ εork > K, stop.The simulation experiments show that when the request of relativeerror is met, compared with the results from [12], [13], momentummethod, this improved BP wavelet networks based on DY ? HS algorithmenhances the network's learning efficiency, and obtains a highcompression ratio, which makes it possible to further simplify thenetwork architecture. When the relative error is no more than 0.0005,we make a comparison among the results from [12], [13], momentum methodand this article. Then we draw the conclusion that the improved BPWN has better performance than others', both on iterative times andlearning time requested.To test the capability of this network further, we let the relativeerror upper limit be 0.00005. In this condition, all the 10-timeexperiments meet this limit when the iterative number is no more than2400. So we try to reduce one hidden neuron. 20-time experiments showthat the new network with 14 hidden neurons also can meet the relativeerror upper limit of 0.0005, and it also performs well.