Object Recognition Using Joint Kernel Sparse Coding Based On Image And Tactile Fusion Method

The most commonly used sensor modes of robot system include: vision, depth,tactile, wrist and so on. In a manner of a single modality or a fusion modality these sensors guide the robot to complete the fine manipulation tasks, depending on the different fields of the application of robots. In the operation of the target object recognition, effective visual information representation and learning were an important prerequisite of obtaining operating information(shape, color, material, size,position and direction), recognizing operation objects, directing operation planning,and therefore visual model plays a leading role in robot target recognition system.However, tactile model of the robot plays an important role in completing fine operations to provide more accurate positioning information and the physical characteristics information of the target object. Through the use of tactile perception to reflect changes in the contact force during contact the tactile model compensates for the lack of rely wrist force sensor, distance sensors, laser radar and other perception ways. Compared with the single-model data processing sensor, the multi-modal sensor makes effective use of complementary sensor information resources in order to obtain a more complete overall goal of detected object and environmental information. Therefore, this article analyzed the visual and tactile single model in recognizing the household objects, and collected a image-tactile data set of the household objects to solve the problem of image-tactile fusion problem on object recognition depending on the fusion classification model.First of all, the visual, tactile, image-tactile fusion technology researches are introduced in the field of intelligent control of robots. The principle, system structure and methods of the three modal sensors are stated. The advantages and disadvantages of the visual and tactile information alone are certified. It leads to the image-tactile fusion theory to a reasonable and innovative object certification issues.Secondly, we use the BH8-280 Barrett Hand and SCHUNK manipulator to collect the tactile information of the objects, in the State Key Laboratory of IntelligentTechnology and System, Tsinghua University. Dynamic Time Warping(DTW) is used to realize coding for the whole tactile sequence. The DTW kernel maps the tactile sequence to a linear high-dimensional space corresponds to the nonlinearity in the original European space. To realize the classification of the object, we develop a joint kernel sparse coding(JKSC) method to fuse different tactile sequences which are captured by different fingers.Lastly, this paper proposes the image-tactile fusion algorithm to test the classification of the object. The multivariate time series model is used to represent the tactile sequence and the covariance descriptor is used to characterize the image.Nearest neighbor classification algorithm is used to fusion the two modal of information. We also develop a practical dataset which includes 18 household objects for verification and the experimental results shows that the performance of image-tactile fusion is obviously better than using single modality.