Ensemble Feature Selection Based On Privacy Preserving

When confronting massive data,feature selection is usually a necessary step for data mining and machine learning.Currently,secure machine learning,especially in privacy preservation,has attracted much attention.However,feature selection with privacy preservation is still a new issue,especially for feature selection related to ensemble learning.Differential privacy is a privacy protection method with strict mathematical theory.In this paper,the differentially private ensemble feature selection algorithm based on output perturbation is proposed FWELL-ENOUT.Meanwhile,considering the sequence of adding noise and result integration,the differentially private ensemble feature selection algorithm based on output perturbation is divided into two algorithms,one is the first perturbation last ensemble(FPLE)and the other is the first ensemble last perturbation(FELP).The differentially private ensemble feature selection algorithm based on objective perturbation(FWELL-ENOBJ)is also proposed.And we prove necessity meeting the definition of the differential privacy and perform experiments to verify the effectiveness of algorithms.The experimental results demonstrated that the FELP has a better privacy preserving performance than the FPLE and the FWELL-ENOBJ has a better performance than FWELL-OBJ(differential privacy feature selection algorithm with no ensemble).In addition,local differential privacy feature selection algorithm is proposed based on another privacy protection strategy(local differential privacy).Besides the theoretical proof,the experimental results also demonstrated their high performance under certain privacy preservation degree.In view of the overall situation,the FWELL-LOCAL has a little better performance than differential privacy feature selection algorithm.