Cross-Media Web Information Retrieval And Adaptive Display For Mobile Devices

In the current information age, a large amount of digital contents are generated, stored, spreading and transformed in the form of multimedia (text, graphic/image, video/audio etc.) and the increase is explosive. Especially the galloping development of Internet promotes the endless emergence of application service and the rapid expansion of multimedia data. The present multimedia information service and content are mostly specifically designed for desktop PCs. In many cases, they are not suitable for devices like Personal Digital Assistants and Smartphones with relatively limited display capability, storage, processing power, and network access.Nowadays, the rising mobile devices are more and more popular in our lives. With the capability to access to Internet, these devices provide users a brand new experience for pervasive information. As a combination of Internet search and mobile communication technologies, mobile search overcomes the limitation that the information can only be retrieved and browsed by a desktop PC. With mobile search technology, users can freely acquire the needed information on-the-go.However, the information access on mobile devices is still not as easy as desktop PCs due to some difficulties. The current information retrieval methods are not equally applied on mobile devices. Mobile search is not a simple duplicate of Internet search on mobiles. Further considerations are required such as accuracy of retrieved results, assistant information for displayed entries and personalized information needs. Usability is also a necessary consideration for a mobile search engine, which should be with an intelligent and friendly user interface.In addition, the other difficulty for the information acquisition on mobiles is how to browse the retrieved information under the low band-width connection and small display. Currently, most of Internet information services publish the media data in a unique form -totally same content for the different users. The objective and context differences among variant terminals are still not taken into account. Therefore, more and more research attention has been drawn on the topics how to make mobile users obtain the Web information in an effective and easy way and how to make the multimedia content structuralized and more adaptive for different devices.The dissertation focuses on facilitating the multimedia information search on mobiles and improving the user experience of mobile search. In addition, we research on the intelligent multimedia content analysis and modeling to achieve adaptive multimedia display and to provide mobile users a friendly interface to access multimedia data. The main content and innovations in this dissertation are as follows: 1. The background, motivations and promising prospect of mobile search are introduced. Researches on mobile search and multimedia content adaptation are also reviewed. We also discuss the main topics and problems on mobile search and multimedia content adaptation.2. We introduce the development and status of mobile Web. The literature of mobile search technology is reviewed according to related ongoing researches. The characteristics, techniques of mobile search are specified and the possible breakthrough is discussed.3. We elaborate on the concept and related techniques of cross-media information retrieval, which is compared with the information retrieval for the single kind of media. The common technical solutions cross-media information retrieval in are also discussed.4. We illustrate the mobile search by the cross-media information retrieval techniques in an example - Photo2search mobile search system. A set of visual search algorithms in large-scale data are proposed in a mobile search scheme for the devices with the embedded camera. The system supports multimodal queries and provides an easy way to carry out visual search on mobiles. The experimental results in large-scale Web data show the effectiveness and efficiency.5. Considering the advantage of multimodal information in the TV program, we propose and implement a Web-like browsing scheme for TV programs. Multimodal information is used to structurize the TV program and to produce the hyperlinks to Web content. An integrated Web-like display interface is provided in an adaptive and zoomable manner, which can be applied to different display sizes.6. A scalable and adaptive image attention model is proposed for heterogeneous devices and universal access. Based on this model, several solutions of adaptive image display are designed. Moreover, we combine the image attention model with video transcoding based on H.264 encoding standard to carry out an effective video adaptation scheme for mobile devices.