Research On Cyberspace Situational Awareness Technology Based On Topology And Traffic Mining

CyberspaceSituationalAwareness(CSA)referstotheacquirement,comprehension,assessment, visualization of the factors which can bring changes in network situationand the forecast of the development trend in the large-scale network. As thedevelopmentdirectionofnetworkmanagementinthefuture,CSAcanfusemulti-sourceand multi-attribute information, assess and forecast the current state and trend of thewhole-network which is composed of the operating status of various networkequipments, network behaviors, user behaviors and other situation factors, and providethe decision support. Currently the research on CSAis just at thebeginning. There aremany problems to be solved: the current research mainly focuses on security, whichcouldn'treflectthesituationcharacteristicsofintegrityandmacroscopy;themainstreamassessment methods are based on hierarchical structure or weight function, which lackthe theoretical basis; most researches remain at the data level, not up to the situationlevel,whichcan'trealizetheabstractfromdatatoinformationagaintoknowledge.According to the typical problems and common requirements of CSA, we studiedthe current key technologies and the application deployment, proposed a CSA model,and mainly researched the network data stream clustering algorithm, situationassessment method and situation forecast method. We also designed and implemented aprototype system to validate our work. The major contributions of this thesis are asfollowing:Considering the shortcomingof traffic analysis and the advantage of data mining,we proposed aCyberspace Situational Awareness model based on Topologyand TrafficMining (TTM). TTM model specifies the CSA functions as well as their division andorganization, defines the data structure, and gives the modeling process and awarenessprocess. The basic idea of TTM model integrates traffic mining and topologyinference,so TTM breaks through the limitations of the security situation, and takes the networktraffic and topology as data source to establish the index system including varioussituation factors which can affect the network situation. TTM provides a higher-levelmore-abstract comprehensive situation, realizes the whole-network assessment andvisualization,and fullyreflects the situationcharacteristics ofintegrityandmacroscopy.In addition, introducing the data mining, TTM is theoretical, scientific and objectivewith the capability of knowledge acquisition, law discovery and known/unknownanomaliesdetection.Aiming at the lack of prior knowledge of situation pattern, clustering wasdetermined as the means oftraffic mining.Analyzing the existing clusteringalgorithmsand thecharacteristics of traffic data, we put forward a network data stream clusteringalgorithm for situation pattern partition -- NetStream. On the basis of clustering space grid partition and situation factor selection, NetStreamfirst merges the connected gridstoformclustersinfull-dimensional space;andthensearchesdenseprojectionclustersinthe clusters unsatisfied density threshold by means of top-down subspace clustering;finallydetects concept drift based onChernoff Bound,dynamically adjusts the windowsize and update interval of jumping windows, and incrementally modifies clusteringmodel. NetStream is a fast subspace clustering algorithm, which can deal withhigh-dimension, burst nature, heterogeneous attributes data and satisfy all of therequirements including: one-pass,ordinal access input data, limitedmemory,scalability,comprehensibility, insensitivity to noise and so on. More importantly, the top-downstrategy, which realizes the fast subspace clustering, takes full advantage of the datadistribution characteristic caused by the burst nature of network, and can find theprojectionclusters with different dimensionalityin different subspaces; theconcept driftdetection based on Chernoff Bound, combining with incremental update strategy, canfind the network burst behavior and realize the online clustering and dynamicalmaintenanceofdatastream.To enhance the theoretical basis of situation assessment, we proposed a SituationAssessment method based on Rough Set Analysis (RSSA). On the basis of situationpattern partition,RSSA generates the situation assessmentrules of the network elementsautomatically through Rough Set analysis; further designs the adjustment strategy forassessmentrules according to the appearancefrequency of situation pattern; meanwhileanalyzesthetopologycontributionandtransmissioncapacityofthenetworkelementstodeterminetheirweights basedonthecapacitynetworktheory; finally fuses thesituationand weight of each network element and completes the whole-network situationassessment. On one hand, with the aid of Rough Set analysis, RSSA integrates theknowledge expression, learning and analysis into a uniform framework, and has theabilityof expression, learning and classification. RSSA has superiorities at the aspect ofdiscovering connotative knowledge, revealing potential law and designing logical rulesfrom massivehistoricaldataorcases. RSSA does not needanypriorinformation,so itisscientificandobjective.Ontheotherhand,withtheaidofGraph Theoryanalysis,RSSAintegrates topology and traffic data, analyzes the effect of network topology structureand network element transmission capacity on the whole-network situationcomprehensively, and realizes the network situation assessment from a globalperspective.Aiming at the problem of nonlinear system forecast, we proposed a SituationForecast method based on Generalized Regression Neural Network (GRNNSF).GRNNSF regards situation forecast as the time series analysis, trains GRNN usinghistorical data, selects network parameters adaptively, and updates the forecast modeldynamically with the arrival of new data. GRNNSF is fast, accuracy, and hassuperiorities in approximation ability, classification ability and learning speed over Back-Propagation Network or Radial Basis Function Network. Even if the sample dataislacking,theforecastresultisalsogood.Tovalidatethesekeytechnologiesdescribedupon,wedesignedandimplementedanetwork situation management prototype system -- NSMS. NSMS integrates twonetwork management functions: topology discovery and traffic collection,puts forwarda multi-view hypervolume visualization scheme, implements NetStream, RSSA andGRNNSF,anddemonstrates TTMmodel.Our research is a beneficial exploration of Cyberspace Situational Awareness. Itprovides essential support to network situation management environment.The researchis valuable to facilitate network management and has been integrated into our actualproject.