The Application Of Information Entropy In Machine Learning Algorithm

Nowadays,people have stepped into the era of big data,and machine learning has become the core technology to deal with the complex information in reality.However,the existing machine learning algorithms still have many shortcomings.There are still many parts of information entropy to be studied in machine learning.Facing the huge amount of information in the stock market,investors are eager to grasp the first-hand effective information and explore the law of the rise and fall of stock prices.Therefore,scholars have intensified their efforts to apply machine learning to financial markets.Taking stock price prediction as an example,it is of great significance to systematically study the application of information entropy in machine learning algorithms for the development of machine learning algorithms,as well as the development of national finance and technology.Some scholars have proposed reinforcement learning algorithms and uncertain data processing methods which are based on information entropy for practical problems.However,the application of information entropy in machine learning algorithms still has a large research space.In this paper,four representative machine learning algorithms such as Decision Tree algorithm,SVM algorithm,BP Neural Network and Hidden Markov algorithm are selected as research objects.This paper also pointed that some existed outcomes,such as information entropy in decision tree algorithm,cross entropy which can be used as loss function in BP neural network,maximum entropy principle which can be combined with Hidden Markov algorithm.This paper also innovatively combines information entropy and existing machine learning algorithm,and proposes a new combined algorithm which is called "information entropy-SVM combined algorithm".In order to facilitate the test and comparison of the performance of each model,this paper designed an experiment to predict the rise and fall of stock prices.The underlying stocks is selected from SH-A share.In this paper,seven machine learning algorithms are used to predict the rise and fall of stock prices of three stocks,and the accuracy of predicted results,F1,the time required for modeling,and the interpretability of the models are taken as evaluation indexes.Through horizontal and vertical comparative analysis among models,the following conclusions are drawn:(1)machine learning algorithm combined with information entropy has better interpretability in practical modeling;(2)information entropy can be used as a tool to analyze and process data sets,so as to effectively simplify the original data set,eliminate redundant information,and improve the prediction performance of the model;(3)in specific application scenarios,it is more appropriate to use cross entropy as the loss function;(4)the maximum entropy principle can be used in the machine learning algorithm to provide a method of processing rules for the algorithm and improve the prediction accuracy of the algorithm.The research of this paper provides a reference for future generations to explore the use of information entropy in machine learning algorithm,and also provides some new ideas for stock market investors to study the law of stock price.