Adaptive Learning Techniques For Few-Shot Classification,Representation And Detection

Modern machine learning approaches achieve remarkable improvements in various real-world fields.The key element is building a complicated model based on large amounts of training data.However,sometimes the task only consists of few data or only directs at few data in the dataset.These few-shot learning problems are more difficult than traditional machine learning problems due to the lack of data.Besides,in realworld few-shot applications,the environment is open and dynamic,which makes the problem more complicated.As a consequence,few-shot learning techniques are still under development.Recently,this field attracts lots of attention from academic and industry community and becomes an important research area.To deal with few-shot learning problems in complex environment,not only should we analyze the underlying difficulty,but also should we propose adaptive techniques to fit the open and dynamic environment.To this end,this paper proposes a series of adaptive learning techniques for different few-shot learning settings:1.Few-Shot Classification with Adaptively Initialized Task Optimizer.Considering the data collection and labeling cost,training a model with limited examples is an essential problem.Directly training a model on few-shot classification tasks falls into the over-fitting dilemma.By treating the few-shot task as an entirety,extracting task-level pattern,and learning a task-agnostic model initialization,the model-agnostic meta-learning(MAML)framework enables the applications of various models on the FSL tasks.Although MAML possesses empirically satisfactory results,its initialization neglects the task-specific characteristics and aggravates the computational burden as well.To solve these issues,we propose adaptively initialized task optimizer(AVIATOR)approach,which incorporates task context into the determination of the model initialization.This task-specific initialization facilitates the model optimization process so that it obtains highquality model solutions efficiently.2.Few-Shot Representation Learning with Adaptively Shifting Embedding.Few-shot representation learning,the problem of learning informative representations for rarely appearing IDs,is important for recommender system.To better understand this problem,we design elaborate experiment on real-world industrial recommender and demonstrate there exists severe frequency bias in embeddings,that is,few-shot IDs and frequent IDs can be distinguished with high accuracy in embedding space.We further give analysis about the biasing factor.The analysis reveals that few-shot ID embeddings are not learned sufficiently and have smaller L2 norm.Motivated by our empirical insights,we address few-shot representation learning with adaptively shifting embedding(SPREAD)approach.SPREAD shifts ID embedding towards its interest prototype,which enables information to spread among IDs of different frequency.Moreover,it utilizes the frequency bias in embedding to adaptively determine the degree of shift,assisting to better represent both few-shot and frequent IDs.3.Few-Shot Multi-View Anomaly Detection with Adaptively Re-weighting Approach.Few-shot anomaly detection aims at identifying anomalies in a dataset.Anomalies in multi-view data are complicated.Specifically,there are two types of anomalies in multi-view data: anomalies that have inconsistent features and anomalies that are consistently anomalous.To detect these anomalies,we propose the nearest neighbor-based multi-view anomaly detection(Muv AD)approach.Specifically,we first propose an anomaly measurement criterion and utilize this criterion to formulate the objective of Muv AD to estimate the set of normal instances.We further develop two concrete relaxations for implementing Muv AD.Muv AD does not rely on the clustering assumption and directly estimating the set of normal instances,which leads to better performance.Moreover,Muv AD is robust to anomalies thanks to the adaptive re-weighting.