| The giant companies of the Internet, such as Google, Yahoo, Amazon and Microsoft and so on, have large amounts of data. The exponential data bring out many problems, so they have to discover the new technologies to anaylyze TB and PB level mass data to achieve useful information. The message is useful to those companies to find the popular books and musics, and recommend the popular news and books to the potential customers. But, the existing tools are becoming unable to handle such large data sets.Google, the first company who provide the MapReduce programming model and the model is able to process the data of PB levels in inexpensive computer clusters parallelly.Hadoop is a representative of the batch processing platform, users can in the case of low-level details did not understand the distributed development of distributed application, its powerful capacity, high fault tolerance and good extensibility, ensures the reliable processing mass data. Hadoop is mainly composed of Distributed File System(the Hadoop Distributed File System, HDFS), graphs of programming model. This article respectively to its working mechanism are briefly analyzed, and the deployment of parallelization of application to the Hadoop platform experiment.Discrete Fourier transform frequency discretization is implemented by using Fourier transform can realize signal spectrum analysis, the filter frequency response time of computing and signal through the linear system of convolution operation, etc. Therefore,the discrete Fourier transform in the field of signal spectrum analysis plays a very important role. But due to the discrete Fourier transform in time complexity in the process of transformation is too big, can’t meet the real-time processing requirements, so later, someone put forward the Courier Fourier transform(FFT). Then, this paper introduces the features of FFT algorithm. Based on the characteristics of cloud computing technology, we to FFT algorithm is improved, and realized based on MapReduc FFT parallel, the parallel FFT algorithm is composed of four graphs work(Job), signal data are filling the Job, the Job of indexing operation, butterfly operation Job, and formatting data output Job JobControUer class sequence operation set up for dependencies.This article finally to Hadoop platform computing performance tests and analyses,by comparing the single test, Hadoop platform to show the characteristics of the real-time performance and high reliability, shows the parallel algorithm is compared with other solutions based on graphs show the advantage. |
PDF Full Text Request