Research On Energy-based Model For Out-of-distribution Detection

In practice,deep models often encounter unknown(Out-Of-Distribution,OOD)samples that significantly deviate from the training distribution,but the model may still identify the sam-ple as some known(In-Distribution,ID)classes with a high confidence level,which can cause serious consequences.Out-of-distribution distribution detection aims to determine whether the input to the model is an ID sample from the training distribution or an OOD sample from some other distribution.OOD detection is critical to ensure the reliability and safety of the model,especially for high-risk application scenarios.For example,in autonomous driving,the system is expected to be able to provide timely alerts and quickly hand over driving control to the driver when an unknown/abnormal scene or object is detected and a safe decision cannot be made.OOD detection has become a key component of security deployment models in real-world applications.This problem was first introduced in 2017 and has quickly attracted wide attention from the research community due to its importance.A growing number of OOD detection meth-ods have been proposed in the industry,including classification-based methods,density-based methods,distance-based methods,etc.The thesis focus on classification-based methods,which refer to the use of a pre-trained DNN classification model to derive a measure of uncertainty from its domain of feature space,gradient space,or output space to calculate the anomaly score of the sample to be tested to distinguish ID samples from OOD samples.The Energy Based Model(EBM)is an unnormalized probabilistic model that models the problem by capturing the interdependence between variables and energy.The energy model calculates the probability density of a sample with an unknown normalization constant.Be-cause this unknown parameter is usually difficult to estimate,training the energy model is very difficult.In the energy-based anomaly distribution detection method,the process of training the energy model is not involved.It is possible to use the energy score to calculate the anomaly score of the sample to be tested and determine whether the input is OOD or not.The theiss adopts the energy-based score to detect OOD samples,which improves the detection performance.Sparse representation tries to express as much information as possible with as little data as possible,which can bring many benefits in high-dimensional space,such as reducing the over-head of operations and preventing overfitting of the model.Current methods for OOD detection usually use the information extracted from the model to calculate the anomaly score directly,which largely ignores the sparsity.The first work in the thesis introduces a sparse module in the learning process and proposes a sparsity-based framework for OOD detection with two opti-mization strategies based on the availability of auxiliary datasets: a sparse regularization method when auxiliary datasets are available,and a sparsity-regularized outlier exposure method when they are not available,respectively.Outlier exposure establishes a new paradigm for OOD detection with the help of some real-existing diverse data as auxiliary anomaly samples.Such methods use a large-scale auxiliary dataset in the experiments,which is so disproportionate compared to the ID training dataset.In an open-world setting,it is difficult to model all possible unknown distributions due to objective constraints.The second work in the thesis constructs a tiny auxiliary dataset,optimizes the experimental setup,and verifies the reliability of the experimental results.In addition,from the information theory perspective,a theoretical insight of the outlier exposure method is given.