The Study On Some Key Technologies Of Random Forests And Deep Neural Networks

The rapid development of artificial intelligence cannot be separated from the support of big data,computing power and machine learning algorithms.This dissertation focuses on machine learning algorithms in artificial intelligence.Specifically,this dissertation selects two very representative algorithms from two popular types of machine learning algorithms,namely,random forest algorithm of ensemble learning and deep neural network algorithm of deep learning.This dissertation addresses the limitations and deficiencies of these two types of algorithms,and conducts in-depth research on theoretical guarantees and practical applications,and proposes corresponding solutions.The main contributions are summarized as follows:(1)To address the problem of greediness and lack of adaptability in decision trees of random forests,this dissertation optimizes the splitting criteria and the method of tree construction from the perspective of Tsallis entropy.First of all,this dissertation proposes a unified Tsallis splitting criterion that unifies the existing decision tree algorithms.Based on the above Tsallis splitting criterion,this dissertation further proposes a symmetric two-term splitting criterion and a maximal-orthogonality-maximal-relevance method for tree construction,which reduces the greediness and improves the adaptability of decision trees.(2)For high dimensional data,the performance of random forests degenerates because of the random sampling feature subspace for each node in the construction of decision trees.To address the issue,this dissertation proposes a combination of feature transformation and stratified sampling method for feature subspace selection in random forests.First,this dissertation theoretically analyzes the reasons for the poor performance of random forests under high dimensional data.Second,this dissertation proposes a random forest algorithm based on feature transformation and stratified sampling.The random forest algorithm proposed in this dissertation has good generalization performance under various data,regardless of low dimensional or high dimensional data.(3)To address the dilemma of the theoretical and experimental performance of random forests,i.e.,random forests with good experimental performance do not have guaranteed theoretical properties and random forests with guaranteed theoretical properties often perform poorly,this dissertation proposes a Bernoulli-controlled random forest algorithm that uses two Bernoulli distributions to help determine the splitting attribute and splitting points used by each node.Specifically,it uses a random process or a deterministic process to construct a decision tree in random forests with a certain probability.Bernoulli random forest algorithm proposed in this dissertation not only has the proved consistency but also has good experimental performance.(4)To address the issue of lacking the interpretability of their performance degradation on data sets with noisy labels for deep neural networks,this dissertation explains the learning process of deep neural network from the perspective of subspace dimensionality.This dissertation uses a dimension metric called local intrinsic dimensionality to analyze the representation subspace of the training sample.This dissertation shows that under the data set with noisy labels,deep neural networks follow a two-stage learning:1)the early dimension compression stage,which simulates the low dimensional subspace closely matching the real data distribution,and 2)the later dimension expansion stage,which gradually increases the subspace dimension to accommodate noisy labels.Based on this finding,we propose a new training strategy called dimensional-driven learning.By adjusting the loss function,we avoid the dimension expansion stage of learning in deep neural networks.(5)For the problem of how to robustly train deep neural networks under complex noisy labels,this dissertation proposes an iterative learning framework that uses iterative noisy label detection,discriminative feature learning,and reweighting modules.These modules can benefit from each other and be jointly enhanced.The proposed framework can not only separate the class categories,but also separate noise samples from clean ones.It does not rely on the noise model and is a more flexible training method of deep neural networks.It can solve the situation of complex open set noisy labels which are often encountered in real-world data.Open set noisy labels refer to that a noisy sample possesses a true class that is not contained within the set of known classes in the training data.