Research On Surface Material Classification Algorithm Based On Reinforcement Learning

Surface material classification plays an important role in remote manipulation and robot recognition.Traditionally,this classification is based on visual data from surface materials.With the rapid development of tactile sensors,various types of tactile data are considered for classifying surface materials.For example,tactile acceleration can reflect the hardness and roughness of surface materials,as a supplement to visual data to develop better classification schemes.When the measured visual tactile fusion data balance the number of samples on different surface materials,it can be effectively classified using common machine learning tools such as convolutional neural networks.However,in most practical scenarios,such as object classification in robot recognition,measurement data for different objects or materials are unbalanced.In other words,the number of measurement data samples representing one material category is different from the number of other material measurement data samples.Existing machine learning based classifiers tend to emphasize primary classes while ignoring secondary classes in the classification process,which inevitably leads to performance degradation.Compared with general classifiers,the strong representation ability of reinforcement learning is more suitable for surface material classification with unbalanced measurement data.The reward mechanism of reinforcement learning can closely link the reward function with the sample number of surface material categories,and automatically adjust the impact of each category(majority or minority)on classifier training.At the same time,some algorithms in reinforcement learning can also avoid over-fitting through asynchronous parameter update.In order to improve the performance of surface material classification methods with unbalanced measurement data,this paper proposes two innovative and practical methods to effectively solve the problem of surface material imbalance classification.The research contents of this paper are as follows:(1)A method based on Res Net50 is proposed to extract low-dimensional visual and tactile mixed feature vectors from unbalanced visual images and tactile acceleration.Specifically,each tactile acceleration sample is first converted into its spectrum through short-time Fourier transform(STFT).Then,input the spectrum of visual image and tactile acceleration into the pre-trained Res Net50,and extract the low-dimensional visual feature vector and tactile feature vector from the output of the global average pooling layer.Finally,we connect each visual feature vector and each tactile feature vector to form a visual tactile mixed feature vector.The numerical results show that compared with the existing feature extraction methods,Res Net50 achieves the balance between classification accuracy and computational complexity.(2)The application of focal loss combined with neural network to the classification of surface materials with unbalanced visual and tactile measurement data is studied.Specifically,the use of focal loss replaces the cross entropy loss used in traditional neural network training.Through a dynamic scaling factor,the focal loss can dynamically reduce the weight of easily differentiated samples in the training process,and improve the proportion of surface material loss function with small number of measured data samples or difficult classification to alleviate the performance degradation caused by the imbalance problem.Compared with traditional classification methods,the method based on focal loss can deal with the problem of unbalanced data classification of surface materials more effectively.(3)A method based on Double DQN(DDQN)is developed to effectively classify surface materials with unbalanced visual and tactile measurement data.Specifically,the visual tactile mixed feature vector extracted by Res Net50 is input into DDQN in the form of state sequence,and an estimated class label is assigned to each state sequence.By comparing with the real label,DDQN gets the reward determined by the unbalanced proportion of the sample number between the material categories.DDQN continuously enhances its classification performance of unbalanced measurement data by maximizing cumulative rewards.It should be noted that,unlike the existing SMOTE and Near Mss technologies,DDQN automatically mitigates over-fitting through rewards and its sub-networks(evaluation network and target network).The experimental results show that the method based on DDQN outperforms the existing methods in the classification of surface materials with unbalanced measured data.