A Workload-driven Approach To Dynamic Data Balancing In MongoDB

With rapid growth of demand on big data storage,MongoDB has been a prevalent choice to store unstructured data in recent years.MongoDB evenly distributes data across shard servers to ensure that all the shard servers hold approximately same amount of data and the data access workload will be balanced across these servers.This approach,however,can hardly guarantee the performance of data access when there are hotspots in data because it supposes all the data will be accessed in same patterns.This paper targeted to optimize the dynamic data balancing architecture of MongoDB in order to improve the performance of storage system.The contributions of this paper can be concluded into three aspects.Firstly,this paper demonstrated the weakness of existing balancing approach in MongoDB when the workload is not evenly distributed across shard servers.The experiment we performed has shown that the imbalanced workload will results in poor performance of data access and low utilization of computing power.Then this paper puts forward a workload-driven approach to dynamic data balancing in MongoDB.In this approach,the workload will be monitored in real time by parsing and analyzing log of MongoDB in order to find the hotspots of data.Then,the heat of hotspots will be diffused across shard servers by dynamic data migration.After migration,the workload will be rebalanced across shard servers though data are probably not even distributed,and the performance of hotspots access will be improved.Finally,this paper implemented this approach as a prototype of tool and demonstrated its effectiveness by comparison between existing balancing approach and our approach.NoSQL database has become the first choice when we store big data.However,NoSQL database does not have a perfect dynamic data balancing strategy since it just got focus of society in recent years.The studies of this paper optimize original strategy of MongoDB and provide a solid foundation for future studying of NoSQL database in dynamic data balancing area.