Application Of Paxos Algorithm In Massive Solar Observation Data Storage

Huge amounts of data has been collected in the procedure of sun observation. All of them should be carefully stored for further research. Saving and processing this data needs a special storage system with features of guaranteeing consistency after write-ops and high availability. No suitable or customized distributed storage system meets all the requirements mentioned above, whose direct reason differs and may turn out to be architecture case, scalability, availability issue or something more.In order to make all observation result preserved effectively, this paper carry lots of related study out. At first we picked file model up to organize all the date. To make out this decision takes various considerations into account which include from format standard to actions taken by image processing algorithm. Then we design a scheme to build an astr-oriented file system called GlobalAstroTable. The scheme fix four major algorithms and reach the goal of making meta-date distributed and more available, replicating date block in a more consistent way, allowing concurrency IO operations by changing lock semantics and providing atomical update function. All this four replaced algorithms are optimized by the idea came up from paxos. At last, we also implement GlobalAstroTable and design a serials of benchmarks run on it. Results tell that the GlobalAstroTable is quite effective and efficient.More detailed works include:1. Making a comparison between three kinds of date model widely used in distributed storage system which include key-value model, file model and scheme-free model. Choosing file model to be the most suitable model for presenting observation data and pointing out that the file model is in line with astronomical researchers’s habit, owning moderate granularity for concurrence control, scale-out, well-studied.2. Moving study into the scalability and fault-tolerance of meta-data, replication algorithm for block-data, concurrency control and commit protocol. Solving them by state-machine replication approach, vertical paxos, paxos lease and paxos commit.3. Designing a new distributed file system called GlobalAstroTable and sharing the creative idea taken part in, like storing meta-data through LSM-tree, a implementation of cow based transaction using paxos commit.4. Verifying the adaption between GlobalAstroTable and observation data from the feature of availability, concurrency, IO semantics, IO Speed, scalability and so on.