Object Hardness Recognition Based On Semi-supervised GAN And Multi-depth Feature LSTM

Object hardness recognition is one of the important tasks in the field of tactile,the purpose is to enable the manipulator to effectively identify the hardness level of the object touched,and to provide important reference information for the next step operation of the manipulator.Object hardness recognition is of great significance in industrial automation,medical robots,special robots and other applications.At present,there are two main problems in object hardness recognition based on deep learning:(1)lack of large-scale labeled open data set of hardness recognition,which restricts the accuracy of hardness recognition based on depth learning;(2)lack of deep learning model for optimization design of hardness recognition task.In view of the above two problems,this thesis mainly includes the following three aspects of research work:1.To solve the problem of lack of large-scale open data set of hardness recognition,this thesis establishes a small hardness recognition database,and then proposes a method based on semi-supervised Generative Adversarial Nets(GAN)to automatically expand the database samples.Firstly,the Unsupervised Training Generative Adversarial Nets(USTGAN)is obtained by training the network with unlabeled hardness recognition samples,and then the parameters of USTGAN are used as the Supervised Training Generative Adversarial Nets(STGAN),and then use a small number of samples with labels to fine tune STGAN,and finally use the generator of STGAN to expand the hardness recognition samples with labels.2.In order to effectively integrate the large-scale generated samples and a small number of manually labeled samples to improve the accuracy of hardness recognition,this thesis proposes a method based on network structure and parameter value sharing to build the Hardness Recognition Nets(HRN).Firstly,the discriminator of USTGAN is taken as the backbone network of HRN and its parameters are shared.Secondly,the classification layer of HRN is redesigned according to the task requirements,and then the HRN is pre trained with the expanded generated samples.Finally,the network is fine-tuned with a small number of manually labeled samples to get the final HRN.3.In the face of the lack of deep learning model for the optimization design of hardness recognition tasks,this thesis proposes a model of the combination of Multi-Input Convolution Neural Nets(MICNN)and Long Short-Term Memory(LSTM).Firstly,starting from the characteristics of periodical changes of hardness recognition data,the same hardness recognition data is respectively sent into the MICNN sub-networks according to different time scales,and the intermediate features of different time scales with time sequence are obtained.Then,these intermediate features are sent to LSTM in turn to obtain multiple one-dimensional features.After splicing these one-dimensional features,a full connection operation is carried out to classify the softmax to obtain the final hardness level.In order to verify the effectiveness of the proposed method,this thesis makes a quantitative evaluation on the classification accuracy,overall classification accuracy,computation time,confusion matrix and kappa coefficient.The experimental results show that the hardness recognition effect of HRN pre-trained by extended samples is similar to that of HRN trained by a large number of manually labeled samples.The hardness recognition result based on MICNNLSTM is obviously better than that trained by a large number of manually labeled samples,which shows the effectiveness of the proposed method.