Design And Optimization Of A Distributed System For Three-dimension Reconstruction

In recent years,with the rapid development of drone technology,aerial market expands sharply.Making use of a drone and the corresponding flight control software,people can get the aerial photos of an object easily,which arouses people's enthusiasm to take three-dimension reconstruction.However,the existing three-dimension reconstruction platform,which is based on stand-alone server,cannot deal with the large-scale reconstruction request from users efficiently.As a result,it is urgent to introduce distributed technology.Besides,in terms of scheduling policy and resource management,the existing platform are fundamentally flawed,which decreases the user experience greatly.The three-dimension reconstruction platform based on distributed system can effectively improve the system's ability to deal with large-scale three-dimension reconstruction.With additional optimizations,the platform can run efficiently and steadily.By analyzing the features of three-dimension tasks,the back-end of the platform implements and integrates a new distributed computing system,which ensures the easy transplantation of reconstruction core codes,without too much modification.To deal with the problem that GlusterFS is not suitable for storage of lots of small files,the system borrows an idea about merging small files so as to design a new Translator.When there are large amounts of reconstruction requests waiting for execution,the system takes both the size and waiting time of a request into consideration,and proposes WERF scheduling policy,which avoids the request's too long waiting.In addition,the system implements a resource manager on Worker node to dynamically limit the resource utilization of reconstruction tasks,which ensures the stability of the platform.System tests includes two parts: functional tests and performance tests.The tests result shows that the distributed system consists of 8 servers can reduce the completion time of a large-scale three-dimension reconstruction to 16.7%.The new scheduling policy can avoid the requests' too long waiting and reduce the average waiting time by 19%.With the new designed Translator,the read/write performance of small files in GlusterFS is improved significantly,which further speeds up the reconstruction.