Personal Identification Based On Video And Audio Feature Fusion

Due to the application limitation and low accuracy of single mode speech recognition and face recognition, this thesis fuses two kind of information on feature level by information fusion theory, which realizes personal identification by audio and video two-mode features.First, we analyses single mode speaker recognition and face recognition. Combined the advantages of VQ and SVM, we provide a mixed speaker recognition model based on VQ and SVM. For the eigenface recognition algorithm, we use L1-norm, Euclidean distance, MIN distance and mahalanobis distance as distance measurement and compare the performance of the four distances. We then propose the face recognition algorithm based on Pulse Coupled Neural Network (PCNN) and build a face recognition system by this algorithm.Second, this thesis studies the recognition algorithm based on audio-video feature fusion. Information fusion on feature level has large amount of available information and can be used in real-time computing. So in this thesis we present two kind of recognition algorithm. One is based on normalization and SVM. The other is based on PCNN. We fuse the audio and video signals on feature level and recognize the speakers in the experiment. For the former algorithm, we use feature connection to combine audio feature and face feature. For the latter one, we fuse the entropy sequence of the two kind signals. Experiment results show that the recognition accuracy of the fusion system is higher than that of the single mode system. When noise is added to the speech signal, there is a dramatic decline on the recognition accuracy of single mode system. But for the fusion system, its recognition accuracy keeps on a satisfactory level.