Research On Key Techniques Of Text Representation Learning For Stock Market Prediction

The stock market has accumulated a large number of structured and unstructured data throughout the years.Big data in the stock market not only provides more abundant and comprehensive information to market participants but also brings new challenges of big data analysis and processing.The traditional models that rely on humans to read and analyze massive data and make investment decisions is no longer realistic in the era of big data.Replacing artificial analysis with machine intelligence has become a major trend.Text representation is the crucial step of intelligent analysis.Previous studies used manually defined feature templates to transform the original data into feature representations that are suitable for the machine learning algorithm.The design and selection of such feature templates are usually time-and labor-consuming,and rely heavily on the domain experts.At the same time,it is challenging to transfer feature templates directly into other domains or languages.Besides,the traditional discrete features usually have higher dimensions(millions of dimensions),so it is difficult to obtain good semantic compositions and text understanding.The trend of the stock market is highly complex,which requires a deep understanding of information.Compared with the model based on discrete features,the text representation learning based on the deep neural network can better represent the text semantics,and has the strong nonlinear fitting ability,and can better describe the relationship between features and learning objectives.How to apply it to the stock market forecast is very worthy of in-depth study and exploration.However,the related work has just started,there are still some problems,such as the prediction model is based on the shallow text understanding,there is no effective fusion of knowledge,the lack of interpretability of the prediction model and so on.To address the above problems,s put forward the research points of this paper,which are as follows:Knowledge-enhanced event representation for stock market prediction The existing event-driven forecasting models are mainly based on a single event,which do not effectively use background knowledge and context knowledge.In order to better represent the implicit semantic information of events,the thesis proposes two stock market prediction models,which integrate knowledge graph and context knowledge,respectively.The experimental results in three different tasks show that compared with discrete-event rep-resentation and event representation independent of external knowledge,the integration of knowledge graph and context knowledge model has made a significant improvement in these three tasks.Target-dependent sentence representation for stock market prediction Eventbased models rely on event extraction tools,and the existing sentence representation model can not adequately capture long-distance dependence between words.The thesis proposes a sentence representation model based on a tree structure network.The model is based on the encoder-decoder structure,learning to explore the possible binary tree structures with reinforcement learning.We reward the structures that can improve the prediction results to obtain a better target-dependent sentence representation for our task.The experimental results on two sentence-level target-dependent prediction tasks show that our model can achieve better prediction results than the baseline methods that operate on parse trees.Besides,the tree structures generated by the model can help us understand how the semantics of sentences are combined according to specific targets.Target-dependent document representation for stock market prediction The existing document representation models are not tailored for a specific target,and the thesis proposes a target-dependent text representation framework in this chapter.The key idea is to learn a target-specific representation of the news abstract,which is then used for aggregating target-specific representations of other sentences in the document.Our framework gives the best performance on two target-dependent document-level prediction tasks.Notably,the advantage of our framework is more apparent when combining information from multiple sources.Interpretable text representation-based stock market prediction: Existing textdriven forecasting models suffers from insufficient interpretability.We propose an interpretable prediction model framework that only relies on document-level(task-level)labels for training.The framework can extract key sentences from the text to explain the prediction results.Experimental results on two different tasks show that our method is superior to unsupervised extraction and existing document representation models.At the same time,the result of model extraction is highly consistent with human annotations.Above all,to address the existing problems in the current text-driven stock market prediction model,this desertion is committed to two aspects of research.On the one hand,the thesis builds the prediction model on text representations of different granularities(event-level,sentence-level,and document-level).On the other hand,the thesis alsostudies the interpretability of the text-driven prediction model.Although our model is mainly concentrated on the stock market,the prediction problems in different fields have some common characteristics.At the same time,in our experiments,we also evaluated our models on the tasks from other fields,and they have achieved relatively consistent results in different fields.