Research On Efficient Machine Learning

Machine learning algorithms provide computers the ability of mining the decision rules from the historical data.Since they avoid the huge effort to summarize the rules like expert systems,these algorithms have been widely used in many real-world applications,such as text mining,computer vision,and recommendation systems.Nevertheless,most machine learning algorithms focus on the effectiveness while ignoring the importance of efficiency,which leads to the extreme computational complexity and becomes impractical for handling big data.Meanwhile,the complex machine learning algorithms involve different types of soft constraints,which is an extra burden for hyperparameter tuning and will waste the computations if the setting is unsatisfied.Therefore,the topic of efficient machine learning attracts more and more researchers,which aims to learn the experience of making decisions from big data in a fast and easy way.Based on the strategies like low-rank decomposition,approximate estimation,and distribution guidance,we perform the research on the efficient machine learning from the following aspects,including the designs of classification models,the selections of to-belabelled samples,and the post hoc explanations of the decisions of black-box classifiers.We study the efficient semi-supervised classifier based on graph models and the fast query selection measures in active learning.Besides,we study the efficient visual explanation for the deep neural networks.Novelties include:1.We develop a fast local weight estimation algorithm to speed up the graph construction.Specifically,it introduces absolutity constraints rather than inequality constraints into reconstruction problems,which leads to efficient analytical solutions for the optimization.Besides,we introduce it into anchor graph classifiers,which becomes 5 to 10 times faster than previous algorithms.2.We propose hierarchical anchor graphs to model adjacency relationships for large-scale datapoints.Based on the pyramidal structure,we introduce the low- rank decomposition for label matrix of datapoints,which decouples the tradeoff between effectiveness and efficiency.We develop two efficient classifiers in input and spectral feature spaces.The experiment results show that these classifiers can perform classification for 8 million samples within 2 minutes.3.We propose approximate error reduction to guide the scalable active learning, which can select high-quality queries 20 to 100 times faster than the one built upon expected error reduction.4.We develop distribution guided explanation for convolutional neural networks, which obtain high performance without good explainability.We introduce distribution controllers into UNet models.The proposed method avoids ad hoc constraints and hyperparameter tuning,speeding up the training significantly.5.We make a survey on efficient machine learning methods from the views of the computational complexities,the computational efficiency,and the computational capability.We additionally provide the guidance for their real-world application.