Learning High-Performance Models From Limited Annotations

Nowadays,machine learning models have shown impressive power in handling various jobs,such as image classification,machine translation and speech recognition.However,when training a well-performed model,numerous data,fast machines and expertise are always required,so machine learning methods are often severely affected when labeled data is scarce and difficult to obtain.In order to reduce the cost of labeling and make full use of unlabeled data,model reuse and semi-supervised learning techniques have been widely used in practice and achieved remarkable results.However,both technologies are still limited for users who hope to deploy learning models in their jobs:1)Model reuse results in a limited performance improvement;2)Semi-supervised learning requires experts to optimize network architecture.This thesis studies these two issues and has made the following new progress.1.In this paper,we propose Ac MR,an active model reuse based rapid performance improvement method.Unlike previous model reuse methods,we consider pre- trained models to help construct queries,facilitating active learner when labeled examples are insufficient on the target task.Moreover,we leverage pre-trained models to filter out not very necessary queries so that considerable queries could be saved compared with direct active learning.At last,the relationships between pre-trained models and the target task are continually updated during the learning process such that they can predict unlabeled samples more accurately.Theoretical analysis verifies that Ac MR requires fewer queries than active learning.Experiments on three transfer learning datasets show the superior performance of the proposed method.2.In this paper,we propose DSSL_AO,a deep semi-supervised learning method based on architecture optimization.The advantages of the proposed DSSL_AOdepends on three folds.Firstly,the superiority of differentiable architecture search is fully absorbed to afford an efficient architecture optimization for DSSL without experts. Besides,we introduce unsupervised loss term for computing architectural gradi- ents,which helps architecture optimization toward a more accurate direction.At last,we propose a better performance measure for neural architectures by adding consistency loss between unlabeled validation data and augmented one,encourag- ing the searched architectures with better generalization.Extensive experiments on two commonly used NASBenchmarks?CIFAR10 and SVHN datasets verify the effectiveness of our proposed DSSL_AOwhich clearly improves performance for DSSL.