Researchon Key Algorithms And Technologies Of Distributed Systems And Het Erogeneous Computing

With the rapid development of big data,cloud computing,and artificial intelligence,more and more enterprises,universities,and government agencies have generated a high demand for the calculation and storage capacity of massive data.On the one hand,data centers need to store massive amounts of data,and on the other hand,they need to provide strong support for computing tasks such as machine learning and deep learning.Because the computing and storage capabilities of a single general-purpose server are limited,distributed storage and heterogeneous computing are becoming the key to managing massive amounts of data in current and future data centers.This thesis aims to study the key algorithms and technologies in distributed and heterogeneous computing environments.Through the combination of theory and practice,this thesis studies and implements the corresponding heterogeneous computing and distributed storage systems,and solves three focus issues on the massive data calculation and storage.The strong computational task of deep learning poses a huge challenge to general-purpose servers.In this thesis,the LSTM(Long short-term memory)algorithm in deep learning is deeply studied,and an FPGA accelerator for LSTM algorithm in cloud heterogeneous computing environment is realized.Most of the FPGA(Field Programmable Gate Array)accelerators of the predecessor's LSTM algorithm are oriented to the embedded environment.This thesis mainly optimizes the FPGA accelerator for the cloud environment of the data center,and obtains a significant improvement in computing and transmission performance.How to ensure high fault tolerance and strong consistency of data is a key issue in distributed systems.This thesis combines Raft algorithm and consistent hash algorithm,and makes some modifications and optimizations to achieve a distributed and consistent file storage system.The system can ensure high fault tolerance and strong consistency of data in extreme environments such as machine downtime and network partitioning.Distributed storage systems need to meet extremely high performance requirements.This thesis studies several performance-enhancing algorithms and techniques to solve the problem of file multi-version management,storage engine,metadata management and data read and write performance for distributed file storage systems,so that the distributed storage system can eventually meet high-throughput and low-latency performance requirements.