Research On The Key Technology Of Processing Large Data Based On Hadoop

With the advent of the information age, a variety of data is growing fast. At present in thetelecommunications industry, the CRUD(Create,Retrieve,Update,Delete) of massive data in the logsystem and user order system still depends on traditional relational database. There are a lot ofproblems, first there is a performance bottleneck of high concurrent read and write, secondly, thecost of data storage is high, furthermore the horizontal scaling of data architecture is very difficultand the data maintenance, data dump cost is large. Hadoop,one of the platforms for cloudcomputing perfect matches the mass data of enterprise application requirements, it become anexcellent solution for this problem.In this paper, first of all, we analyze the key technical problems and main performance andcharacteristic of cloud computing, then analyze the system architecture of Hadoop which is anopen-source cloud-computing platform. We specifically focuses on Hadoop distributed file system,Map/Reduce programming model,HBase(Hadoop database).Combined with the status of the big data of the telecommunication of Jiangsu province, weanalyze the detail technical requirement and design the global interface of big data; we analyze thedata of customer orders, the credits of customer account, the data logs processing and made detaildesign for all modules. we also realize the system with code.Finally, we set up a development environment, then the simulating experiments are conductedfor verifying the feasibility and superiority of the proposed approaches.