Research And Applications Of Activation Function In Monte Carlo Method Based Neural Network

Recent development of artificial neural network has made it a hotspot in artificial intelligence.And as an essential component,activation function has attracted more attentions than ever before.Researchers are creating and modifying current activation functions in order to improve the artificial neural network in use,in both scientific fields and in practice.However,existing activation functions normally apply single mode,while the scenario of combining multiple functions and its analysis remains to be developed.For different activation functions,their non-linear expression abilities differ,especially in the precision of artificial neural network and the training time consumed.Therefore,it is applicable to combine the advantages of diverse activation functions and eventually improve the prediction of artificial neural network.Based on the aim proposed,this paper includes the analysis of the combination of multiple activation functions based on Monte Carlo neural network,considering its simple network structure and the flexibility of the training.Also,the research is illustrated with case analysis including the prediction of artificial neural network,regression and classification,and compares the impacts of five activation functions,Sigmoid,Tanh,Gauss,Softplus,ReLU,and their weighted combinations to artificial neural network with respects to electricity load forecasting,ECG lead reconstruction,and breast cancer classification.The paper proposes an optimized combination activation function based on Genetic Algorithm which has fine global search capability,and conducted comparisons amount the three above-mentioned experiments and six other activation functions.Our results indicate that compared with single activation function,the combination activation function optimized by Genetic Algorithm shows improved precision on electricity load forecasting,ECG lead reconstruction,and breast cancer classification.