Information Fusion And Object Recognition Technology Based On Sound And Acceleration Sensors

Sensors are the bridge between robots and the world.The research on sensors is becoming a new hotspot with the development of artificial intelligence.How to make robots recognize various objects accurately depends not only on the optimization of the sensor structure and the improvement of the sensor function,but also on the analysis and processing of the data collected by sensors.One of the difficulties in the research is how to deal with the data of different modalities.Image information sometimes cannot represent the features of an object completely.This inadequacy can be made up by the addition of acceleration and sound information.In this thesis,the experimental subjects are acceleration,sound and image information.The sensor and artificial intelligence algorithm are combined to design a recognition and retrieval system based on the sound and acceleration sensor which lays a foundation for the research of information fusion and object recognition of robot sensors.The main work is as follows:Firstly,we summarized the research background and status of information fusion and object recognition technology,and introduced the significance of using sound and acceleration sensors.Then the internal structure and working principle of the microcontroller,acceleration sensor and microphone were analyzed.The hardware was embedded in the body of a 3D printed pen,which formed a device for collecting experimental data.A data set containing acceleration,sound and image information of 20 different wood samples was established based on the obtained experimental data.Secondly,we extracted the Mel Frequency Cepstral Coefficient(MFCC)feature of the sound signal and the Power Spectral Density(PSD)feature of the acceleration signal.The image feature was extracted by Local Receptive Fields based Extreme Learning Machine(ELM-LRF).The principle and extraction process of each method were elaborated,and the results of feature extraction were given.Then the algorithms of K-Nearest Neighbor(KNN),Support Vector Machine(SVM)and Extreme Learning Machine(ELM)were used to classify the features.The parameter settings and optimization results of each method were given.The confusion matrix in the optimal classification was drawn to analyze the recognition accuracy.Experimental results show that the ELM has a better recognition effect than other algorithms for the established data set.Finally,the acceleration and sound features were separately combined with the image features using Canonical Correlation Analysis(CCA)and its extension method.The combined features were used for cross-modal retrieval.The mean average precision and precision-recall curve were used to evaluate the retrieval performance,and the retrieval results were verified by experimental comparison.Some similar images of the wood were obtained from the acceleration or sound signals which were collected by the experimental device acting on the surface of the wood.A graphical user interface was designed to display the retrieval results.The Cluster Canonical Correlation Analysis(CCCA)method was used to solve the problem of the non-one-to-one corresponding relationship between variables in the correlation analysis.Experiments show that CCCA can make feature fusion better and improve the performance of retrieval.