Application Of OLAP In Video Site Log Analytics

Online video has become the next hot Internet application after web portals and searching engines. Not only should the video website provide high-quality video content and best user experience, but also be able to analyze the operational data in order to make decisions in time.Timeliness of information is important to enterprise in the age of Internet. Video website also requires analysis software capable of real-time multi-dimensional analytics.In this paper, several existing web analytics technologies will be discussed and proven to be unable to suffice video website, as they do an imprecise and coarse statistics based on page view in a lazy manner.After presenting the requirement of video website, this paper propose a solution to video website real-time operational data analytics based on data warehouse. The first step in the solution is defining a new mechanism for user track. By means of registering some predefined events on client application, the mechanism enables the client to send message to log server for statistics. Different from the traditional web server log which is records of page access, the log file on the log server is records of user behaviors and events of video plug-ins, and is the source of precise analytics.Data staging area in this solution is capable of real-time ETL. The set of visitor sessions is maintained according to a finite state machine.In visitor session, there is a dataset describing user behavior and video plug-in information. The finest fact data called 'basic fact data' will be calculated by sampling of visit sessions per minute, and used to populate the data warehouse.In data presentation area, a dimensional model is defined conform with requirement of video website. An OLAP engine gives a portal to up-to-date dimensional data.Basic fact data is sort of streaming data with the feature of huge volume, flowing in-and-out dynamically and changing rapidly. The challenges in OLAPing streaming data are maintenance and storage of data warehouse.In order to solve these problems, stream cube is introduced into this solution. Because tilt time frame can not keep the fluctuation information of dimensional measure, we propose a time frame considering the fluctuation of dimensional measure. A partial materialization plan is given based on this novel time frame. It is proven that this materialization plan can relieve the pressure of data maintenance and storage, while keep the valid and useful information for analysis.