| This is an era about users, for example, web search engine personalized reranking, digital advertisement targeting and various recommendation systems. By engaging with various online platforms, users generate huge amount of data. AI is striving with these data and many opportunities arise. Nevertheless, there are still big challenges, for example, label scarce, cross domain, multi-modality and label ambiguity, before we can fully take advantage of these weakly supervised data. We develop models and algorithms to learn from various aspects of user interactions with images and videos.;To demonstrate the usefulness of learning from web weakly supervised data, we develop applications to link YouTube videos with Wikipedia entities, to describe videos with natural languages, to estimate air quality from photos, and to discover and curate fashion outfits.;Built upon successes in practice, we further develop models and theories that are specifically designed to handle the intrinsic challenges of learning from noisy image and video data. In particular, we design a pairwise ranking loss function and label decision model to learn convolutional neural networks from multi-label images, and we show their effectiveness both theoretically and empirically. Furthermore, we develop a framework based on knowledge distillation to learn from noisy labels, and we show its effectiveness theoretically and empirically on a large scale image dataset with real world label noise. |
