Study Of Multimodal Integration And Based On Intelligent Information Processing Methods

The aim of user interface research is to provide an efficient way of communication for the user. With the quick development of computer software and hardware and the new introduction of Internet, the traditional user interface faces new challenges. So the multimodal user interface is come forth. Multimodal interface allow users communicate with computer system with collateral and blurry multi-interface to increase the naturalness and efficiency.As a new form,it can reinforce the integration with reciprocal modal,each modal can complement the limitation each other,or employ different communicate manner for different application,so the robustness and the flexiblity of a system is increased.The aim of multimodal interface is to get the user's intent by more than one sense and action modals and to make the interaction of human and computer more natural and more effective. The key of multimodal interface is multimodal integration. Modern user interfaces can take advantage of multiple input modalities, such as speech, gestures, handwriting, to increase expression ablity, naturalness, flexibility and mobility.The paper discusses the key problems related to the research of multimodal interface , that is multimodal integration. The paper introduces the main integration method. The paper mainly studies the integration theory and the integration arithmetic about every method.The integation method based on statistics can make use of rear probability , on the other hand,the integation method based on syntax analyse is able to describe the complex syntax structure.The author construct the experimental condition with mature discriminating method for speech and gesture and makes experiments for every method, at the end of this paper,the author realize a new integation method-method based on Finite-State Transducers,this method colligate the main strongpoint of other methods.