Research On Table Learning Method Combined With Language Model

Tabular can orderly store and display the multi-dimensional elements that affect decision-making,and it is an effective tool to help managers make decision.Looking at the existing analysis methods for tabular data,the following problems deserve attention: the current analysis methods are only aimed at tables containing categorical features and numerical features,and cannot analyze tables containing subjective description fields;machine learning occupies a large part of the field of tabular data mining.However,the increased dimension of decision-making factors will affect the analysis results.Because of the above problems,it is particularly important to explore the rules of tables that contain numerical features,categorical features,and subjective description text features at the same time for decision-making.This paper proposes a tabular learning method based on deep learning and combined with language models and conducts regular exploration and evaluation of the results of the questionnaire on the sustainable development of engineering at the North China University of Technology.In this method,the subjective description fields are converted into text similarity and text opinion sequences respectively through the language model,and they are spliced with other features in the table and then sent to the tabular model for analysis and decision-making.The main work of the paper is as follows:(1)For the subjective field of the concept,the paper uses TF-IDF to extract the keywords and word frequency of the students' answers and uses the Jaccard coefficient as the evaluation index;the paper uses Word Embedding to extract the vector space of the students' answers and the official text and does a similarity comparison,they describe the concept space together.(2)Aiming at the subjective description field to check whether students have correct opinions,this paper proposes ML?BERT,which takes the BERT language model as the framework,and fine-tunes the model structure and loss function to complete the multi-label opinion extraction task,and proves its effectiveness through experiments.(3)Aiming at the problem of label imbalance in the multi-label text opinion extraction task,this paper introduces multi-label Focal loss to optimize the classification task.(4)In this paper,the language model is combined with the tabular learning model,and Tab Net is modified for the opinion sequence to avoid feature dispersion and focus more on the subjective answer content.Experiments show that the end-to-end tabular learning model selected in this paper is more suitable for combining with language models than other methods,and has interpretability.