| With the almost ubiquitous presence of computing power and connectivity we are currently experiencing a rapid growth of the amount of available digital information. For example the data warehouses of large retailers are already managing dozens of Terabytes of business data. Similar trends occur in digital libraries, geographical information systems, and scientific databases. Analyzing large databases involves a high amount of summarization and aggregation. Data cubes have been proposed to combine an intuitive data model with efficient processing of aggregate queries. We have developed a novel framework that configures data cubes for a large variety of applications with heterogeneous attributes and different requirements for query and update performance. For large data warehouses whose contents evolve over time we propose another technique that supports efficient integration and aggregation of historical information. This technique takes advantage of the append-only nature of updates which occur in many practical applications, e.g., business, statistical, and GIS databases. Furthermore we propose aggregation techniques which are particularly designed to scale to data warehouses with sparse data sets and large numbers of attributes, e.g., by taking advantage of the high transfer rates of modern hard disks. This work has been conducted in the context of the Alexandria Digital Earth Prototype (ADEPT) project, a digital library initiative that involves multiple departments and universities. |
