Research On Massive Neural Structure Data Management Method Based On Distributed Framework

The human brain,as the highest organ,controls the daily activities of the human body.Humans have never stopped exploring the brain.Researchers use different tools to study the brain from different aspects,hoping to understand the mechanism of the brain.Using electron microscopy to image sequence section of the brain tissue and then reconstructing the connectome is also one of the current research directions in the preface,and also faces many challenges.Recently,with the development of advanced technology and the expectation of reconstructing complete connectome,the reconstructed brain tissue grows as well as the capacity of data it generates.The challenge is how to effectively manage large-scale data so that researchers can reuse and share it among users to mine valuable information.For this purpose,a data management module is developed,composed of two parts,the storage and retrieval module on the server-side and the image cache module on the client-side.On the server-side,combining Hadoop and Hbase,the pyramid model is adopted to store electron microscope images to solve the problem of massive data storage.And a fast retrieval method is designed based on the spatial location with constant time complexity O(1).A method based on block storage is proposed to store electron microscope image segmentation,as well as two retrieval methods for acquiring segments by layers and for getting a whole object,respectively.On the client-side,a three-level image cache module is designed to reduce the delay time of data acquisition further and improve the experiences of users.Through theoretical analysis and the practical test,our tool shows excellent real-time performance,can support large-scale data and is scalable.