Research And Optimization Of Cloud Storage Gluster Fs For Security Video Surveillance

With the development of the urban city scale,the construction of safe city and wisdom city becomes a top priority.Meanwhile,banking,airport monitoring,shopping mall place a greater demand on security video surveillance,such as high-definition,networking.The work mode of video surveillance is 7*24,so it is normal to lead to the explosive increase of video files.Facing the explosive increase of video surveillance data and large-scale data clusters,the cloud storage with virtualization,scalable and distributed storage is growing up to take the place of traditional storage architecture.By comparing the Lustre,Ceph,HDFS,GlusterFS four open source distributed file systems,GlusterFS is better than the other three file systems in scalability,high availability,high reliability,so,we choose GlusterFS as the model of security video monitor distributed file to provide cloud storage service.The design of no metadata server architecture avoids the single point of failure or performance bottlenecks,its replication volume provides mutilple copy technology.On the contrary,the no metadata server design shows poor performance in directory operations.GlusterFS should have a balance between high efficiency file operations and high scalability.Through in-depth study of the GlusterFS underlying implementation and handling process,we summarized the bottleneck of its directory operations,the dentry cache in Linux kernel based on semantic is put forward,and the dentries will be loaded in memory to improve the performance of directory operations.The scheme using the Linux kernel provides some caching mechanism,break the traditional caching mechanism unable to identify the semantic cache data limitations,according to the cache data type(high value metadata and video data)under different caching strategies,to improve the problem of a directory tree structure.File system without metadata architecture,such as GlusterFS,should be a corresponding metadata caching mechanism.GlusterFS modular architecture makes it can dynamically scale each feature,based on that,we optimized the directory entry to read-ahead function in the Readdir-Ahead Translator,designed a directory entry cache function module in dentry-cache Translator.In the design of the dentry-cache Translator,contradictions in the keep of cache and the consistency of cache always exist.Due to the particularity of the application,we will keep as the first priority of cache,in terms of consistency,we use GlusterFS weak consistency was achieved by ageing strategy.By optimizing the performance of GlusterFS directory operations has improved dozens of times,and users get the result of directory operations within 1s,it will better meet the demand of security mass surveillance video storage...