Challenges and opportunities in building personalized online content aggregators

The emergence of "Web 2.0" services has attracted a large number of users to publish content on the Web as blogs, social bookmarks, customer reviews, and wiki articles. Due to the explosion of this "user-generated" content, the amount of new data on the Web is growing rapidly, at a rate that is several times higher than what was believed before. To help users keep up with the continuous stream of new content, many Web 2.0 sites provide RSS feeds that return recently updated materials based on a user's request. Since many users often "subscribe" to a large number of RSS, which may be on the order of hundreds, there is a need for a new online service that helps users manage their growing subscription lists and the constant stream of new content.;In this dissertation, I study the challenges and opportunities in building a large-scale personalized online content aggregator. In particular, I address the following three challenges in building such a system: (1) A significant portion of user-generated content is updated frequently, often several times a day, and is related to current world events whose significance deteriorates rapidly over time. I propose an effective RSS-feed retrieval algorithm based on the updating pattern of the feeds and the access pattern of the users. The algorithm helps the system deliver fresh content to the users in a timely manner even in a resource-constrained setting. (2) In order to help users navigate the continuously updated new content, it is important to provide a service that can prioritize and recommend what the user is most likely to be interested in based on the user's personal interest. I propose a learning framework that efficiently captures a user's interest through an interactive process. I also develop an efficient approach to computing such "personalized recommendations" that can scale well to a large number of users without putting an unreasonable strain on computing resources. (3) The data collected from such a system over time, often in the form of social annotations, involves lots of human effort in matching the best descriptive keywords with the corresponding Web resources. As an example of how this rich body of data can be mined to help users, I analyze its evolution over time and propose methods that discover the behavioral properties of evolving vocabulary usage. I illustrate how these properties can be used to help users select the best keywords in the online advertising context.