Research On Stock Market Hotspot Concept Mining Based On Topic Model And Entity Recognition

With the development of the Internet,and the popularity of social networking platforms such as Twitter and Weibo,an emergent event will rapidly form a hot topic on the Internet.If it's a topic related to the stock,it will often form the stock concept,and the topic related stocks also refer to the concept Shares,unexpected events will have a huge impact on the stock price of the shares.Analysis of the incident and related stocks often require professional industry researchers,ordinary investors can not accurately determine which stocks may be affected.So if there is a complete system that can timely detect the generation of the topic,record its development process,and can identify the concept corresponding to the topic,and then based on the massive Internet information,accurately dig out the event concept stocks,it will bring the investors great help.In response to the above problems,this thesis uses the topic model to achieve the stock market hot topic detection and tracking algorithm.This thesis uses the classical Latent Dirichlet Allocation algorithm to implement the topic detection,and introduce the combined keyword extraction which can help to remove noise words.For the selection of the number of clustering topics,the thesis uses the minimum inter-class similarity as criterion,this way makes topics have big differences with each other and topics are interpretable.At the same time,this article puts forward the topic heat calculation method,which can record the topic development process.The concept of stock is usually a term that is extremely related to the topic.There is no obvious regularity in the structure of the concept,which is often a new word or a specific field word,the word segmentation algorithm can't identify effectively.In this thesis,we combine the naming entity recognition algorithm and the word rough segmentation algorithm to identify the stock concept.First,we study the named entity recognition method based on statistical methods and the naming entity recognition method based on neural network.The neural network entity recognition method with introduction of boundary entropy and word embedding achieves good results on the public corpus.Then a new word finding algorithm based on word rough segmentation is proposed in this thesis,which is used to identify the concept names.Finally,this thesis designs and implements the Web-based stock market hot topic detection and concept stock extraction system.The system includes topic detection and topic trend monitoring,concept stocks and concept trend monitoring module to help investors make decision.