| Various networks and protocols constantly emerge and evolve with the continuously extend of Internet,therewith present varies of protocols according to different requirements.In managements of network systems,those unknown protocols cause analysts huge inconvenience.For work such as anomaly detection,the reverse analyzing of network protocol specifications loom large.With these requirements,the reverse engineering of protocols steps in,getting much attention and developments from analysts.There are mainly two kinds of protocol reverse analyzing methods according to difference of available analysis objects,which are network data-oriented and terminal procedure-oriented protocol reverse.Although theory of the latter one is relatively mature,limits of analysis object in many scenes makes it impractical.Whereas,the former one attracts much attention for its advantage on widespread application,high automation,et al.But many problems exist in present research such as generality of analysis methods,accuracy of results,recovery degree of target protocol specifications,and expression ability of models,which limit the analytical ability and quality of results.This paper researches on network data-oriented protocol reverse analyzing by relative techniques in data mining.To increase the efficiency and effectiveness of analytical methods and reduce the dependence on priori knowledge at the same time thereby improving the generality of reverse engineering methods,this paper study intensively on a series of bottleneck issues including classification based on message formats,general format fields analysis,behavior modeling and inference,and the main researches are as follows:1.An unsupervised message clustering method based on format similarity is proposed to cluster the hybrid traffic of unknown protocols.In measure of format distance between unknown protocol messages,the format similarity of tokens and messages are measured by rational grammar labels and optimizing of sequence alignment algorithm.For the parameter tuning problem in the clustering process,two unsupervised indexes of clustering quality are introduced to assist adjusting of algorithm parameters to cast off manual intervene in this process.Results of experiments prove the performance advantage of this strategy with more stable results on multiple datasets with different distributions.2.A fix-length field segmentation method based on information entropy is proposed as a general method to partition and recognize fields in message head.By measuring relationship between data on neighboring offsets using mutual information saturation and self-information,the distributions of data on different offsets are analyzed and a segmentation method is proposed for head fields partition.Together with recognition of delimiters,the labels of state related fields are extracted for stateful protocols under some conditions.Experiments show the effectiveness of this segmentation method and accuracy of the state label extraction method.This work greatly promotes relative research on general format analysis and extraction of state keywords.3.A novel Stochastic Protocol Transducer(SPT)is proposed to describe the target model of protocol behaviors with the ability of integrating message exchanging and statistic information.An inference method targeted on SPT model is also proposed to reverse analyzing behaviors of unknown protocols,which could relieve the overgeneralization problem existing in result models of behavior reverse analyzing.This model could record message exchanging behaviors in protocol communication process and is equipped with the ability to predict succeeding behaviors.Targeted on the SPT model of analyzed protocol,process of the inference method includes: compensation of state label sequences,prefix-tree building,and recognition and merge of compatible states in the reduced behavior model.When simplifying the prefix-tree,rational merging rules between compatible states are designed to control the degree of generalization of result models.Experiments show the priority of proposed method on higher accuracy,larger coverage,and loser computation cost.4.An optimized SPT learning method is proposed to reconstruct the behavior model progressively to solve the state explosion problem in protocol behavior inference.In learning process of SPT,the compatible states are merged right after their addition.This keeps the constructed SPT model in best state and close to the simplest result model thereby avoiding the emerge of state explosion problem.In experiments compared to last inference method,this progressive method greatly reduces the computation cost of behavior reversing at the expense of tiny accuracy,and this superiority becomes more obvious when dealing with large-scale data. |
