Data Acquisition And Analysis On Web2.0 Application Based On P2P Networks And Social Networks

The emergence and the rising of Web 2.0 brought a revolution to the internet in both its core content and external applications.It also changed the way of using Internet.As typical applications of Web 2.0,P2P networks and social networks play more and more critical roles,also including information and data.The file sharing,streaming media and search engines based on P2P networks accounted for the vast majority of the Internet traffic.Also,using various social networks becomes the main reason to surf the internet.The data acquisition is the first step in many researches on Web2.0,such as analyzing network performance,optimizing user experience and improving network security.The active acquisition is mostly applied for the P2P network and social network.Starting from a certain peer,active acquisition traverses each peer in network.The active acquisition enjoys the advantage of goal oriented,high data parse ability and less environment requirements.However,on one hand,as the network grows,the active acquisiton is subject to the timeliness and data effectiveness as it has to traverse a large amount of peers.On the other hand,active acquisition can not obtain real-time behavioral information such as user's loging in.In order to solve these problems,we proposed a passive acquisition mechanism.Based on the network monitoring,data packet restoring and protocol reassembling,we designed the passive acquision system for BT protocol and twitter with libpcap library.The function and performance of the system are verified by experiment.By comparing the active acquisition and passive acquisition in twitter,we further proposed a joint active-passive acquisition method using the user ID as the linker.Three different joint active-passive acquisition mechanisms are proposed to implement the twitter data acquisition with a certain target and certain topic.These method are verified by experiment.