| The significance of cache memory in bridging the speed gap between CPU and main memory is becoming more useful to data-intensive applications. The complexities and diversity of frequently changing workloads in big data environment have raised considerable concerns and challenges for organizations and researchers. Data storage systems are evolving to meet unique challenges of the present big data era. Cache optimization in the memory hierarchy plays an important role in the current generations of data storage systems. The rapid evolution of big data systems requires a holistic approach to mitigate its needs.;This thesis explores the impact of big data applications on system resources by introducing a cache memory dashboard program, which will run alongside a Hadoop Map-Reduce analytic job. In a single-node cluster, cache activities in the NameNode and DataNode are closely monitored and compared for misses and hit rates. The test results are then used to visualize performance metrics on a dash board.;Based on some useful parameters, this thesis also provides an in-depth comparative analysis on 15 caching techniques employed by researchers and data storage vendors in an attempt to reduce cache miss rate, increase hit rate, and thus enhance storage performance. This thesis leverages Linux Perf utility tool capabilities to get other useful hardware events. The dashboard developed in this thesis is an intuitive, and dynamic web page designed using HTML and JavaScript language. It picks values for different system performance metrics at regular time intervals, and displays them to the user. |
