Research On System Cconstructicon Based On Emerging Topic Detection And Prediction Method

In the background of rapid development of financial technology,the research and application of data analysis technology and machine learning algorithm in financial market become popular and active.We propose research on system construction based on emerging topic detection and prediction method,taking news data as the main research object,and taking the stock prediction as the research topic.Our research problem mainly includes the following two parts:first,digging out the potential valuable information from the massive news text data;second,providing a new exploration direaction for the research of market prediction method with the help of data analysis technology and machine learning algorithm.For the first problem,we propose an system construction method based on emerging topic detection.This method uses HDP for topic modelling and introduces novelty index and related volume index for news data,and further the topic-word distribution is combined with the calculation of detection points and detection values.Finally,an emerging topic detection system for massive news data is constructed.Through empirical research on the Internet news data,it proves that this method can effectively obtain emerging topics and help people analyze the development trend of these topics,which solves the problem of poor interpretability of current topic models and the difficulty in predicting the topic trend.For the second problem,we design a classification prediction model that integrates market data with news text data.The market data includes historical transaction data and technical indicators,and the news text data corresponds to the topic features obtained by emerging topic detection.The model takes the above three types of data as input,and selects three classification algorithms,namely,Naive Bayes,Support Vector Machine and XGBoost to predict the rise and fall of stock price in the in the following trading day.In addition,we propose an analysis framework including basic analysis method,technical analysis method and subject analysis method by studying different input data sources.On the one hand,it helps to compare the prediction results of different classification algorithms.On the other hand,it can further analyze the impact of news data sets from different fields on market prediction of of different stocks.Based on the proposed research framework,we take the accuracy and F1 score as the main indicators to evaluate prediction results.It is proved that the classifiers incorporating the emerging topic features can achieve the best prediction effect,and thus the rationality and effectiveness of our research methods can be demonstrated.