Symbiotic Forest:A Lightweight Decision Tree Ensemble Method

In recent years,the development and popularization of artificial intelligence and machine learning have widely affected various fields in production and life such as computer vision,machine translation,big data analysis,etc.However,despite their powerful performance,these intelligent models are still difficult to deploy on smallscale hardware platforms such as mobile smartphones due to the high storage and computing costs.Too large and redundant intelligent models not only occupy a large amount of storage space but also reduce computing efficiency and even affect prediction accuracy.Based on the classic random forest classification algorithm,this paper proposes a lightweight decision tree integrated pre-pruning algorithm guided by information theory,which is called Symbiotic Forest(SymForest).The specific work is as follows:First,from the perspective of mutual information entropy,this paper analyzes the global impact of the decision tree's node changes on the algorithm learning process,and theoretically proves that the structural redundancy of the tree model increases monotonically with the node split,and then introduces the merge strategy of tree nodes,and proposed adaptive decision tree growth criteria.Secondly,this paper designs and implements an ensemble algorithm based on adaptive growth criterion—Symbiotic Forest,and introduces the detailed process of the algorithm,optimization strategy and other implementation details.In view of the shortcomings of the non-flexible node splitting criterion and many hyperparameters in the algorithm,we simplified and improved it and obtained an improved symbiotic forest algorithm(SymForest 2),making it more suitable for small-scale computing devices with limited memory Deployment.Finally,this paper collects a large amount of classification data in different fields and different characteristics,designs rich verification experiments,compares the advantages and disadvantages of the symbiotic forest algorithm with the random forest,gradient boosting tree,and other tree integration algorithms,and obtains a comprehensive analysis and evaluation for symbiotic forests.The experimental results show that the symbiotic forest algorithm significantly reduces the structural redundancy of the model and improves the learning ability of the model.