The Collect And Filter Of IDS Data

With rapid growth of the Internet Technology, Network Security has become more important and complex each day. Intrusion Detection System(IDS) is an automatic Network Security Defense System. It can effectively make up the shorting of the fire wall. It's the fire wall's second line of defence.This paper mainly focused on data collecting of the IDS, discussed many ways of different types of data collecting, trimming and differentiating. Created an IDS and made it work.This paper analyzed data from multiple sources and different structures, concluded three types of info: Login info, Event info, and Packet data info, then created the formation for every type of data; In order to combine future possible new data, IDS using profile and formation string together, create and practice data formation standard algorithm; Because in the collected data, there is info that is redundant or has minor effect on the IDS, this paper discussed the rules of redundant data differentiating and safe data differentiating, Then create data filter rule base; Data collecting system uses distributivity design, collecting module like a black box. we can get data which was filterd and had standard formation, if we created a new model string for new data source. Module work individually; filter data right away at the collecting node. It only reports suspicious data to analysis center, in order to reduce the amount of data for transfer and store, lower the effect on Network capability.IDS data collecting system is completed using c language under Linux. Collect and filter Linux log, Apache log, Network data package, sent filtered data to the analysis center, save to the MySQL.Today studies of IDS mainly focus on analyze what kind of data can uncover hacking activities. This paper from reducing data's angle analyzes data, filter redundant and safe data, and only report the suspicious data.Redundant rules are:1. If the user's login time and the next login time is within 1 minute, then these two login records would combine as one.2. If the user login from localhost has two records, it would only save one.3. Delete login record from messages.4. Unit time port IP address - destination IP: Ports have same protocol packages, combine as one record, save time range and amount of the packages.Safety rules are:1. All of root user's activities records.2. Certain user's certain activities.3. Reroot event.4. System kernel's activity.IDS data collecting system gets data from the data source, first exchange data into standard formation, then using rule to detect, if accord with any particular rule, then it would delete or combine the data with related data. The algorithm used in this paper is based on Linux platform, but it's compatible with other operation systems. The C language's ability to transplant decided the realistic data collecting system has practical use with other operating systems.