Research On Novel Deep Forest Models

Deep learning is a hot research topic in machine learning,which has achieved great success in various learning tasks.As a new type of deep models based on decision trees,deep forest opens the door to deep learning based on non-differentiable modules,demonstrating the possibility and effectiveness of building deep learning models without backpropagation.This paper tries to expand the new research direction of deep forest to different learning scenarios,including the following aspects:1.Deep forest based on confidence screening.For the learning problems with medium and large-scale samples,this paper proposes a deep forest model gcForestcs based on confidence screening.By introducing a predictive confidence estimation mechanism,it provides direct channels for the prediction of high-confidence samples,thereby significantly reducing the model complexity.The proposed gcForestcs reduces the time cost and memory requirements by an order of magnitude,which greatly increases the sample size that deep forest can handle.2.Deep forest based on feature screening.For the learning problems with highdimensional features,this paper proposes a deep forest model gcForestfsbased on feature screening.By introducing a feature screening mechanism,it selects informative features at each level of the cascade.Combined with an unsupervised feature transformation process,it significantly reduces the computational cost.The proposed gcForestfsreduces the required computing resources by two orders of magnitude,which greatly increases the data dimension that deep forest can handle.3.Deep forest based on comparative information.For the learning problems with inaccessible features,this paper proposes a deep forest model CBDforest based on comparative informations.By introducing a new type of random forest MixForest as building blocks,it can use both triplet comparisons among instances and rerepresentation of instances to realize layer-by-layer processing.Empirical results in various settings validate the effectiveness of our proposal.4.Deep forest for anomaly detection.For anomaly detection tasks with unavailable label information,this paper studies the detection of anomalous users in the recommender system.There are two types of anomalies,i.e.,unorganized and organized anomalies.First,we propose a matrix completion-based method UMA for unorganized attacks detection.Furthermore,we propose a deep forest model IDForest,which uses the output information of multiple detection methods to detect both organized and unorganized anomalies.Empirical results validate the possibility and effectiveness of constructing deep forests in an unsupervised setting.