Design And Implementation Of Distributed Data Processing Algorithm Based On Tensor Train

In the context of the information age,data has become particularly important,with different types of data emerging endlessly.The rapid growth in data volume makes it difficult for single machine processing efficiency to meet the needs of modern applications.Therefore,distributed data processing has become a necessary choice.One of the key issues in distributed data processing is how to effectively partition and allocate data to ensure processing efficiency and data quality.At the same time,the speed of data growth is also increasing,and how to handle the rapidly increasing data is also an urgent problem that needs to be solved.In this context,this thesis uses parallel processing ideas to study the tensor train decomposition problem,mainly focusing on the following three aspects of work:First,a block parallel SVD based tensor train decomposition algorithm(BPTTD)is proposed.The traditional tensor train decomposition algorithm requires the storage of a large number of intermediate results and computational processes,resulting in low efficiency of the algorithm,especially for high-dimensional large-scale data,which seriously hinders the operation of the algorithm.In order to solve the problems faced by the tensor train decomposition algorithm mentioned above,this thesis designs a new tensor train decomposition algorithm.This algorithm first divides tensor data into column blocks,and then sends the data to different computing cores of the computer for processing,making the amount of data that can be processed more and improving the cache hit rate.By formulating a reasonable data allocation strategy,the execution efficiency of the algorithm is improved.The experimental results show that compared with traditional tensor train decomposition algorithm,the algorithm proposed in this chapter can process larger tensors and has good parallel efficiency.Second,a modal growth based incremental tensor train decomposition algorithm(MGITTD)is proposed.Compared with traditional algorithms,BPTTD algorithm has some advantages,such as smaller time complexity and larger processing tensor scale.However,it also has certain limitations,such as difficulty in processing incremental data and upper limit scalability.In order to solve these problems,this thesis first adopts a distributed system to implement the algorithm,which has good scalability.Then,an incremental SVD algorithm and a modal based tensor expansion algorithm are designed to process incremental data.Finally,a merge tree for singular value decomposition from subtensor block is designed to merge intermediate results.The experimental results show that the algorithm has good scalability and can stably process incremental data.Third,this article is based on the distributed data processing method based on tensor train,and designs and implements a complete bearing fault diagnosis system based on tensor train decomposition.It also elaborates on the module composition and model architecture of the system.Firstly,the overall architecture of the system and the functions of each level were introduced,and then the specific functions of the system were introduced according to different functional modules.Finally,experiments were conducted on the system,and the results showed that the system can effectively diagnose faults in bearing data,and can output diagnostic results and fault types.