| Complex data analysis often requires queries that are ad hoc in nature and that involve several underlying base relations. Such queries involve highly complex aggregations and join operations and, as a result, those queries are frequently very difficult to formulate and express in languages such as SQL. Poor expression of complex queries also results in inefficient execution plan and performance. To overcome these challenges in formulating complex queries, people often turn to customized ad hoc solutions that are tailored for specific instances of the problems. However, those types of solutions are often difficult, if not impossible, to verify the correctness and the maintenance of such solutions are very expensive.; This thesis presents a relational operator that provides a framework for complex data analysis. The new operator is highly flexible and comprehensive, yet is based on a fundamentally sound framework---the relational model. This new operator, the MD-Join, provides a clean separation between group definition and aggregation computation, which, in turn, provides great flexibility in expressing highly complex OLAP (OnLine Analytical Processing) queries in a succinct manner. In addition, we present a simple and highly optimizable implementation of the operator. The thesis also examines how the new operator is easily integrated into the existing relational framework using several algebraic transformations.; Additionally, this thesis examines how the MD-Join operator has proven to be highly effective in a more recent research work such as data stream management. It demonstrates how the main idea of the new operator has proven to be highly adaptive in new and challenging areas such as network data analysis, providing a framework to implement solutions for complex data analyses in real time. Specifically, we have defined a new aggregate operator for network analysis purpose: running window operator. We also examine a well known window type, called sliding window. Through a formal mapping between the MD-Join operator and the two window aggregation operators, we will show that those window operators are also relational and demonstrate how these window aggregation operators can be used to formulate ad-hoc queries to perform advanced network monitoring and analyses in real time. |
