Research On Open Set Image Recognition

The close set hypothesis is the hypothesis employed by most machine learning algorithm,assuming the possible object classes in the test set are the same as those in the training set.However,when the model is applied to the real world,the model obviously needs to have the ability to recognize the object classes that do not exist in the training set and to label them as "unknown".A common way to deal with "unknown" classes is to add "background" class or "other" class in the model training stage,so as to provide a certain ability to identify "unknown" class for the model.However,the "background"class can not be exhausted.When there are unknown classes which do not belong to the"background" class during real world application,the behavior of the model is still unpredictable.This is a very dangerous thing to real world application such as automatic driving.Although we can train such a system with terabytes of data in the training phase,we are still unable to predict and exhaust all possible inputs.However,for such a security critical system,even a small number of errors on unknown inputs can be fatal.Most real data is dynamic in nature,and the world is unpredictable.The system must have the ability to process new input or new class,so that the system can ignore or reject them,that is,the system should have the ability of open set recognition.In this paper,the following three aspects have been conducted to solve the problem of open set recognition.1.Find and prove that the compact abating probability model(CAP)is not suitable for the open set recognition algorithm which employ the logits layer of convolutional neural network as its feature extractor and proposed two ways to mitigate the inadaptability.Most of the convolutional neural network based open set recognition models employ the CAP model as their basis.The CAP model assumes that the probability of any test sample being judged as a known category decreases with the increase of the distance between the test sample and the known samples.This is feasible in the traditional open set recognition algorithm based on machine learning,because the feature space used in the algorithm usually does not have directionality.However,the logits layer of the convolutional neural network,as the output layer of the convolutional neural network,has a strong directionality in its feature space,that is,the farther the sample moves in a certain direction,the higher the probability of the sample being judged as a certain type.Therefore,CAP model is not suitable for the open set recognition algorithm with logits layer of convolutional neural network as feature extractor.There are two ways to mitigate the inadaptability:(1)The space of logits layer is transformed so that the boundary generated by cap model can contain samples whose maximum component is much larger than other components.(2)Find a ideal high dimension hyperplane in the space of logits layer,which can divide the space between known space K and open space O.2.According to the first way,OpenSoftMax method is proposed.By making softmax transform to the space where the logits layer of convolutional neural network is located,the transformed space generates boundary.After transformation,directionality still exists in the space,but there is an upper limit on the possible positions of samples,which reduces inadaptability.Comparing to OpenMax,the average optimal accuracy gain of the OpenSoftMax method is 15.55%,and the average optimal F1 score gain is 28.35%.3.According to the second way,OpenMax with Partial Enclosed Boundrary is proposed(OPEB).Since CAP model is the basis of the convolutional neural network based open set recognition method,the shape of generated boundary in feature space should be circle,sphere or hypersphere.However,in a directional space,a reasonable interface should be a semi closed curve,surface or hypersurface.OPEB gives a method to fit the ideal hypersurface.Comparing to OpenMax,the average optimal accuracy gain of the OPEB method is 13.17%,and the average optimal F1 score gain is 25.7%.