On the BP Algorithm: A Regularization Approach 

Author  LiuYang 
Tutor  ZhangDaQing 
School  Liaoning University of Science and Technology 
Course  Operational Research and Cybernetics 
Keywords  BP Algorithm Regularization Method Simulated Annealing Algorithm 
CLC  TP183 
Type  Master's thesis 
Year  2012 
Downloads  64 
Quotes  0 
BP algorithm is thought as the most proven algorithm by its maximum adopting and most widely applying among all the neural network algorithms. It has the advantages that easily operating, good parallelism, and small calculation efforts which are concerned by general scholars as hot topic for studying. However, as the research discovery, the BP algorithm is still short of convergence rate and easily falling into local minimum point etc. More than decreasing the abilities of BP neural network, those disadvantages block its(BP NN) development. So, the improving process for BP neural network is profoundly meaningful for future.Regularization method is provided to solve the morbid state problem which reflected by BP neural network. Regularization is brought for tolerance function, as it could efficiently overcome the random noise or the morbid state problem which caused by data tolerance. Through the simulated case proving, this process could increase the convergence rate of BP neural network, and strengthen the stabilization of network.BP neural network will be easily immersed into local optimum, simulated annealing(SA) algorithm could efficiently solve this problem. But once the SA algorithm is adopted as the improvement process for BP algorithm, the training speed of the whole block will be slowed down. Hereby, this paper is presenting one improvement process for this problem, which is base on the SA improvement process, the regularization is added to the algorithm. The regularization could overcome the morbid state BP neural network problem, and simulated annealing(SA) could effectively solve the problem that BP neutral network will be easily immersed into the local minimum value. So the combination of two algorithm methods could easily solve the morbid state problem, be easily immersed into local minimum point, and meanwhile, it will increase the convergence rate and strengthen the stabilization of the network. Particular analysis is obtained from the theory testing, the effectiveness of this algorithm is proved by the resulted simulated consequence.