1. INTRODUCTION
1.1 Security Concerns
Despite nearly universal efforts to protect corporate networks, todayâ„¢s distributed organizations are still susceptible to a multitude of attacks. IT executives are challenged to extend security beyond the corporate backbone to protect a variety of potential vulnerabilities, including Internet connections, communication channels between remote and corporate offices and links between trusted business partners. Unfortunately, the preventive measures employed to secure corporate resources and internal traffic donâ„¢t provide the breadth or depth of analysis needed to identify attempted attacks or uncover potential threats across the organization.
1.2 Network Security Management
Security is the process of staying informed. The goals of security include Confidentiality (ensuring only authorized users can read or copy a given file or object), Control (only authorized users can decide when to allow access to information), Integrity (only authorized users can alter or delete a given file or object), Authenticity (correctness of attribution or description), Availability (no unauthorized user can deny authorized users timely access to files or other system resources), and Utility (fitness for a specified purpose).
Network Security Management is a process in which one establishes and maintains policies, procedures, and practices required for protecting networked information system assets. The various tools & steps used today for maintaining corporate network security are indicated in Fig.1.
Fig.1 Information Security Market
Any security technology is based on a layered architecture called the Security Hierarchy. The security policy and standards form the foundation of this hierarchy over which other layers like security architecture & processes, security awareness & training, the technology & product and finally auditing, monitoring & investigation, which contribute to overall security.
1.3 Why firewalls are not enough?
Firewalls act as a barrier between corporate (internal) networks and the outside world (Internet), and filter incoming traffic according to a security policy. Thus, a firewall provides a good amount of security lest sufficient protection due to the following facts:
1. Not all access to the Internet occurs through the firewall.
Users, for a variety of reasons ranging from naiveté to impatience, sometimes set up unauthorized modem connections between their systems connected to the internal network and outside Internet access providers or other avenues to the Internet. The firewall cannot mitigate risk associated with connections it never sees.
2. Not all threat originates outside the firewall.
A vast majority of loss due to security incidents is traced to insiders. These include the users who misuse privileges or impersonate higher privileges. The firewall only sees traffic at the boundaries between the internal network and the Internet. If the traffic reflecting security breaches never flows past the firewall, it cannot see the problems.
Organizations utilize strong encryption mechanisms to secure files and network connections. In securing the network from the outside threat, the threat from within the network is almost completely forgotten. Intrusion detection systems are the only part of the infrastructure that is privy to the traffic on the internal network. Therefore, they will become even more important as security infrastructures evolve.
3. Firewalls are subject to attack themselves.
Firewalls are not completely foolproof. A firewall generally makes pass-deny decision on the basis of allowable network addresses. Intelligent firewalls may analyze the contents of packets of certain protocols but they may only identify the anomaly related to that protocol.
A common attack strategy is to utilize tunneling to bypass firewall protections. Tunneling is the practice of encapsulating a message in one protocol (that might be blocked by firewall filters) inside a second message. Thus the inside message gets through as the firewall considers outer, encapsulating message harmless.
In order to strengthen the security, one cannot rely on any single tool. Hence a firewall must be complemented by Intrusion Detection Tools.
1.4 Intrusion Detection Systems
1.4.1 Definition
Intrusion Detection is the unrelenting active attempts in discovering or detecting the presence of intrusive activities. It refers to all processes used in discovering unauthorized uses of network or computer devices. This is achieved through specifically designed software with a sole purpose of detecting unusual or abnormal activity. Such software is called Intrusion Detection System.
1.4.2 History of IDS
The original idea behind automated ID is credited to James P. Anderson who, in 1980, published a study outlining ways to improve computer security auditing and surveillance at customer sites. This paper paved the way to development of misuse detection for mainframe systems.
Between 1984 and 1986, Dorothy Denning and Peter Neumann researched and developed the first model of a real-time ID. This prototype was named the Intrusion Detection Expert System (IDES). This IDES was initially a rule-based expert system trained to detect known malicious activity. This same system has been refined and enhanced to form what is known today as the Next-Generation Intrusion Detection Expert System (NIDES).
During the last 2 decades, numerous projects like Discovery, Haystack, Multics Intrusion Detection and Alerting System (MIDAS), Network Audit Director and Intrusion Reporter (NADIR) were all developed to detect intrusions.
1.4.3 Why do we require IDS?
To answer this question, we need to understand why intruders can get into the system. There are various reasons of which the prominent ones are:
* Software bugs “ they can be buffer overflows, unexpected combinations, unhandled inputs, race conditions etc. Software has bugs because programmers cannot track down and eliminate all possible holes.
* Password Cracking “ hackers have over the time developed numerous ways to break into systems by knowing passwords that were really weak, or by making dictionary & brute force attacks.
* Design flaws “ many systems that were developed early were never designed to handle the wide scale intrusion that is there today. These include TCP/IP protocol flaws, operating system flaws etc.
* Sniffing unsecured traffic “ traffic on the Internet is not encrypted. Hackers can use programs that can get sensitive information from packets over the network. These include the packet sniffers, port scanners etc.
A firewall cannot always handle attacks directed to exploit these flaws. Hence we require IDS which can logically complement the firewall.
2. CLASSIFICATION OF
INTRUSION DETECTION SYSTEMS
There are two ways to classify Intrusion Detection Systems.
2.1 Classification by Monitoring Approach
2.1.1 Application based IDS
Application-based intrusion detection sensors collect information at the application level. Examples of application-level include logs generated by database management software, web servers, or firewalls. With the proliferation of Web-based electric commerce, security will increasingly focus on interactions between users and application programs and data.
Advantages:
¢ This approach allows targeting of finer grained activities on the system (e.g. one can monitor for a user utilizing a particular application feature.)
Disadvantages:
¢ Applications-layer vulnerabilities can undermine the integrity of application-based monitoring and detection approaches.
2.1.2 Host based IDS
A host based IDS resides on the system being monitored and tracks changes made to important files and directories. It takes a snap shot of existing system files and matches it to the previous snap shot. If the critical system files were modified or deleted, the alert is sent to the administrator to investigate. The example of the host based IDS can be seen on the mission critical machines, that are not expected to change their configuration.
Host-based intrusion detection started in the early 1980s before networks were as prevalent, complex and interconnected as they are today. In this simpler environment, it was common practice to review audit logs for suspicious activity. Intrusions were sufficiently rare that after-the-fact analysis proved adequate to prevent future attacks.
Host based intrusion detection tools normally employ agents that must to be installed on the key systems that are to be protected. These agents must be custom built for each platformâ„¢s hardware and software version, and their function is to continuously monitor host-generated logs. The agents monitor the state of the system and various kernel structures to verify the integrity of the system.
Todayâ„¢s host-based intrusion detection systems remain a powerful tool for understanding previous attacks and determining proper methods to defeat their future application. Host-based IDS still use audit logs, but they are much more automated, having evolved sophisticated and responsive detection techniques. Host based IDS typically monitor user and file activity, file accesses, changes to file permissions, attempts to install new executables (including Trojan horses) and attempt to access privileged services. Log files like security logs on Windows NT and syslog in UNIX environments are monitored. When any of these files change, the IDS compares the new log entry with attack signatures to see if there is a match. If so, the system responds with administrator alerts and other calls to action.
Host-based IDS have grown to include other technologies. One popular method for detecting intrusions is to check key system files and executables via checksums at regular intervals for unexpected changes. The timeliness of the response is in direct relation to the frequency of the polling interval. Some products listen to port activity and alert administrators when specific ports are accessed. This type of detection brings an elementary level of network-based intrusion detection into the host-based environment.
One of the main benefits of host based IDS is that it does not have to look for patterns. It only checks for changes within a specified set of rules. Most intrusion detection systems include default policies for specific operating systems. These policies vary with the design of the system being monitored. An administrator can use this information upon initial installation to learn the behaviors of files and directories under normal system activity and enable him or her to fine-tune the policy through trial and error.
Advantages:
¢ Systems can map problem activities to a specific user id
¢ Systems can track behavior changes associated with misuse
¢ Systems can operate in encrypted environments
¢ Systems can operate in switched network environments
¢ Systems can distribute the load associated with monitoring across available hosts on large networks, thereby cutting deployment costs.
¢ Systems require no additional hardware.
Disadvantages:
¢ Network activity is not visible to host-based detectors
¢ Running audit mechanisms can incur additional resource overhead
¢ When audit trails are used as data sources, they can take up significant storage
¢ Operating system vulnerabilities can undermine the integrity of host-based agents and analyzers
¢ Host-based agents must be more platforms specific, which adds to deployment costs
¢ Management and deployment costs associated with host-based systems are usually greater than in other approaches.
Example of host based IDS are Symantecâ„¢s Intruder Alert and Purdue Universityâ„¢s Tripwire (developed by Dr. Eugene Spafford and Gene Kim).
2.1.3 Network based IDS
Network based intrusion detection systems use raw network packets as the data source. A network based IDS typically utilize a packet sniffer, using network interfaces or adapter running in promiscuous mode to monitor and analyze all traffic in real-time as it travels across the network.
There are two main forms of NIDS which are common in commercial products which are in use today. The first is the ËœRawâ„¢ pattern matching NIDS which are designed to do a comparison to the packets they capture and match attacks based on the data captured. This style of NIDS can be considered a Ëœpacket grep[1]â„¢ NIDS, examples being Snort or Dragon. Alternatively, a ËœSmartâ„¢ NIDS can interpret the packet, and attempt to understand the protocol that is being captured in order to identify. ISS RealSecure is an example of a Smart NIDS.
Another variant of NIDS is Network Node Intrusion detection system (NNIDS) “ it performs the analysis of the traffic that is passed from the network to a specific host. The difference between NIDS and NNIDS is that the traffic is monitored on the single host only and not for the entire subnet.
Advantages:
¢ The data come without any special requirements for auditing or logging mechanisms; in most cases collection of network data occurs with the configuration of a network interface card.
¢ The insertion of a network-level agent does not affect existing data sources.
¢ Network-level agents can monitor and detect network attacks. (e.g., SYN flood and packet storm attacks) by checking the content of both the packet header and payload.
¢ Network based IDS use live network traffic for real-time attack detection. Hence attacker cannot remove the evidence, as against host based IDS, where hackers know very well how to manipulate audit logs to remove their evidence.
¢ They are not dependent on host operating systems as detection sources.
¢ Real time detection and response can terminate any malicious activity, as against host based IDS, where an attack is not recognized until a suspicious log entry is written.
Disadvantages:
¢ Although some network-based systems can infer from network traffic what is happening on hosts, they cannot tell the outcome of commands executed on the host. This is an issue in detection, when distinguishing between user error and malfeasance.
¢ Network-based agents cannot scan protocols or content if network traffic is encrypted.
¢ Network-based monitoring and intrusion detection becomes more difficult on modern switched networks. Switched networks establish a network segment for each host; therefore, network-based monitors are reduced to monitoring a single host. Network switches that support a monitoring or scanning port can at least partially mitigate this issue.
¢ Current network-based monitoring approaches cannot handle high-speed networks.
2.2 Classification by Timing of Information Collection & Analysis
2.2.1 Batch or Interval Oriented IDS
In batch-oriented (also called interval-oriented) approaches, operating-system audit mechanisms or other host-based agents log event information to files and the intrusion detection system periodically analyzes these files for signs of intrusion or misuse.
Advantages:
¢ They are well suited to environments in which threat levels are low and single-attack loss potentials high (e.g., financial institutions).
¢ Batch mode analysis schemes impose less processing load on systems than real-time analysis, especially when collection intervals are short and data volumes are therefore low.
¢ Batch-oriented collection and analysis of information are particularly well suited to organizations in which system and personnel resources are limited.
¢ Attacks on computer systems often involve repetitive attacks on the same targets.
Disadvantages:
¢ Users will seldom see incidents before they are complete.
¢ Aggregation of information may consume more disk storage on the analysis system.
1.1 Security Concerns
Despite nearly universal efforts to protect corporate networks, todayâ„¢s distributed organizations are still susceptible to a multitude of attacks. IT executives are challenged to extend security beyond the corporate backbone to protect a variety of potential vulnerabilities, including Internet connections, communication channels between remote and corporate offices and links between trusted business partners. Unfortunately, the preventive measures employed to secure corporate resources and internal traffic donâ„¢t provide the breadth or depth of analysis needed to identify attempted attacks or uncover potential threats across the organization.
1.2 Network Security Management
Security is the process of staying informed. The goals of security include Confidentiality (ensuring only authorized users can read or copy a given file or object), Control (only authorized users can decide when to allow access to information), Integrity (only authorized users can alter or delete a given file or object), Authenticity (correctness of attribution or description), Availability (no unauthorized user can deny authorized users timely access to files or other system resources), and Utility (fitness for a specified purpose).
Network Security Management is a process in which one establishes and maintains policies, procedures, and practices required for protecting networked information system assets. The various tools & steps used today for maintaining corporate network security are indicated in Fig.1.
Fig.1 Information Security Market
Any security technology is based on a layered architecture called the Security Hierarchy. The security policy and standards form the foundation of this hierarchy over which other layers like security architecture & processes, security awareness & training, the technology & product and finally auditing, monitoring & investigation, which contribute to overall security.
1.3 Why firewalls are not enough?
Firewalls act as a barrier between corporate (internal) networks and the outside world (Internet), and filter incoming traffic according to a security policy. Thus, a firewall provides a good amount of security lest sufficient protection due to the following facts:
1. Not all access to the Internet occurs through the firewall.
Users, for a variety of reasons ranging from naiveté to impatience, sometimes set up unauthorized modem connections between their systems connected to the internal network and outside Internet access providers or other avenues to the Internet. The firewall cannot mitigate risk associated with connections it never sees.
2. Not all threat originates outside the firewall.
A vast majority of loss due to security incidents is traced to insiders. These include the users who misuse privileges or impersonate higher privileges. The firewall only sees traffic at the boundaries between the internal network and the Internet. If the traffic reflecting security breaches never flows past the firewall, it cannot see the problems.
Organizations utilize strong encryption mechanisms to secure files and network connections. In securing the network from the outside threat, the threat from within the network is almost completely forgotten. Intrusion detection systems are the only part of the infrastructure that is privy to the traffic on the internal network. Therefore, they will become even more important as security infrastructures evolve.
3. Firewalls are subject to attack themselves.
Firewalls are not completely foolproof. A firewall generally makes pass-deny decision on the basis of allowable network addresses. Intelligent firewalls may analyze the contents of packets of certain protocols but they may only identify the anomaly related to that protocol.
A common attack strategy is to utilize tunneling to bypass firewall protections. Tunneling is the practice of encapsulating a message in one protocol (that might be blocked by firewall filters) inside a second message. Thus the inside message gets through as the firewall considers outer, encapsulating message harmless.
In order to strengthen the security, one cannot rely on any single tool. Hence a firewall must be complemented by Intrusion Detection Tools.
1.4 Intrusion Detection Systems
1.4.1 Definition
Intrusion Detection is the unrelenting active attempts in discovering or detecting the presence of intrusive activities. It refers to all processes used in discovering unauthorized uses of network or computer devices. This is achieved through specifically designed software with a sole purpose of detecting unusual or abnormal activity. Such software is called Intrusion Detection System.
1.4.2 History of IDS
The original idea behind automated ID is credited to James P. Anderson who, in 1980, published a study outlining ways to improve computer security auditing and surveillance at customer sites. This paper paved the way to development of misuse detection for mainframe systems.
Between 1984 and 1986, Dorothy Denning and Peter Neumann researched and developed the first model of a real-time ID. This prototype was named the Intrusion Detection Expert System (IDES). This IDES was initially a rule-based expert system trained to detect known malicious activity. This same system has been refined and enhanced to form what is known today as the Next-Generation Intrusion Detection Expert System (NIDES).
During the last 2 decades, numerous projects like Discovery, Haystack, Multics Intrusion Detection and Alerting System (MIDAS), Network Audit Director and Intrusion Reporter (NADIR) were all developed to detect intrusions.
1.4.3 Why do we require IDS?
To answer this question, we need to understand why intruders can get into the system. There are various reasons of which the prominent ones are:
* Software bugs “ they can be buffer overflows, unexpected combinations, unhandled inputs, race conditions etc. Software has bugs because programmers cannot track down and eliminate all possible holes.
* Password Cracking “ hackers have over the time developed numerous ways to break into systems by knowing passwords that were really weak, or by making dictionary & brute force attacks.
* Design flaws “ many systems that were developed early were never designed to handle the wide scale intrusion that is there today. These include TCP/IP protocol flaws, operating system flaws etc.
* Sniffing unsecured traffic “ traffic on the Internet is not encrypted. Hackers can use programs that can get sensitive information from packets over the network. These include the packet sniffers, port scanners etc.
A firewall cannot always handle attacks directed to exploit these flaws. Hence we require IDS which can logically complement the firewall.
2. CLASSIFICATION OF
INTRUSION DETECTION SYSTEMS
There are two ways to classify Intrusion Detection Systems.
2.1 Classification by Monitoring Approach
2.1.1 Application based IDS
Application-based intrusion detection sensors collect information at the application level. Examples of application-level include logs generated by database management software, web servers, or firewalls. With the proliferation of Web-based electric commerce, security will increasingly focus on interactions between users and application programs and data.
Advantages:
¢ This approach allows targeting of finer grained activities on the system (e.g. one can monitor for a user utilizing a particular application feature.)
Disadvantages:
¢ Applications-layer vulnerabilities can undermine the integrity of application-based monitoring and detection approaches.
2.1.2 Host based IDS
A host based IDS resides on the system being monitored and tracks changes made to important files and directories. It takes a snap shot of existing system files and matches it to the previous snap shot. If the critical system files were modified or deleted, the alert is sent to the administrator to investigate. The example of the host based IDS can be seen on the mission critical machines, that are not expected to change their configuration.
Host-based intrusion detection started in the early 1980s before networks were as prevalent, complex and interconnected as they are today. In this simpler environment, it was common practice to review audit logs for suspicious activity. Intrusions were sufficiently rare that after-the-fact analysis proved adequate to prevent future attacks.
Host based intrusion detection tools normally employ agents that must to be installed on the key systems that are to be protected. These agents must be custom built for each platformâ„¢s hardware and software version, and their function is to continuously monitor host-generated logs. The agents monitor the state of the system and various kernel structures to verify the integrity of the system.
Todayâ„¢s host-based intrusion detection systems remain a powerful tool for understanding previous attacks and determining proper methods to defeat their future application. Host-based IDS still use audit logs, but they are much more automated, having evolved sophisticated and responsive detection techniques. Host based IDS typically monitor user and file activity, file accesses, changes to file permissions, attempts to install new executables (including Trojan horses) and attempt to access privileged services. Log files like security logs on Windows NT and syslog in UNIX environments are monitored. When any of these files change, the IDS compares the new log entry with attack signatures to see if there is a match. If so, the system responds with administrator alerts and other calls to action.
Host-based IDS have grown to include other technologies. One popular method for detecting intrusions is to check key system files and executables via checksums at regular intervals for unexpected changes. The timeliness of the response is in direct relation to the frequency of the polling interval. Some products listen to port activity and alert administrators when specific ports are accessed. This type of detection brings an elementary level of network-based intrusion detection into the host-based environment.
One of the main benefits of host based IDS is that it does not have to look for patterns. It only checks for changes within a specified set of rules. Most intrusion detection systems include default policies for specific operating systems. These policies vary with the design of the system being monitored. An administrator can use this information upon initial installation to learn the behaviors of files and directories under normal system activity and enable him or her to fine-tune the policy through trial and error.
Advantages:
¢ Systems can map problem activities to a specific user id
¢ Systems can track behavior changes associated with misuse
¢ Systems can operate in encrypted environments
¢ Systems can operate in switched network environments
¢ Systems can distribute the load associated with monitoring across available hosts on large networks, thereby cutting deployment costs.
¢ Systems require no additional hardware.
Disadvantages:
¢ Network activity is not visible to host-based detectors
¢ Running audit mechanisms can incur additional resource overhead
¢ When audit trails are used as data sources, they can take up significant storage
¢ Operating system vulnerabilities can undermine the integrity of host-based agents and analyzers
¢ Host-based agents must be more platforms specific, which adds to deployment costs
¢ Management and deployment costs associated with host-based systems are usually greater than in other approaches.
Example of host based IDS are Symantecâ„¢s Intruder Alert and Purdue Universityâ„¢s Tripwire (developed by Dr. Eugene Spafford and Gene Kim).
2.1.3 Network based IDS
Network based intrusion detection systems use raw network packets as the data source. A network based IDS typically utilize a packet sniffer, using network interfaces or adapter running in promiscuous mode to monitor and analyze all traffic in real-time as it travels across the network.
There are two main forms of NIDS which are common in commercial products which are in use today. The first is the ËœRawâ„¢ pattern matching NIDS which are designed to do a comparison to the packets they capture and match attacks based on the data captured. This style of NIDS can be considered a Ëœpacket grep[1]â„¢ NIDS, examples being Snort or Dragon. Alternatively, a ËœSmartâ„¢ NIDS can interpret the packet, and attempt to understand the protocol that is being captured in order to identify. ISS RealSecure is an example of a Smart NIDS.
Another variant of NIDS is Network Node Intrusion detection system (NNIDS) “ it performs the analysis of the traffic that is passed from the network to a specific host. The difference between NIDS and NNIDS is that the traffic is monitored on the single host only and not for the entire subnet.
Advantages:
¢ The data come without any special requirements for auditing or logging mechanisms; in most cases collection of network data occurs with the configuration of a network interface card.
¢ The insertion of a network-level agent does not affect existing data sources.
¢ Network-level agents can monitor and detect network attacks. (e.g., SYN flood and packet storm attacks) by checking the content of both the packet header and payload.
¢ Network based IDS use live network traffic for real-time attack detection. Hence attacker cannot remove the evidence, as against host based IDS, where hackers know very well how to manipulate audit logs to remove their evidence.
¢ They are not dependent on host operating systems as detection sources.
¢ Real time detection and response can terminate any malicious activity, as against host based IDS, where an attack is not recognized until a suspicious log entry is written.
Disadvantages:
¢ Although some network-based systems can infer from network traffic what is happening on hosts, they cannot tell the outcome of commands executed on the host. This is an issue in detection, when distinguishing between user error and malfeasance.
¢ Network-based agents cannot scan protocols or content if network traffic is encrypted.
¢ Network-based monitoring and intrusion detection becomes more difficult on modern switched networks. Switched networks establish a network segment for each host; therefore, network-based monitors are reduced to monitoring a single host. Network switches that support a monitoring or scanning port can at least partially mitigate this issue.
¢ Current network-based monitoring approaches cannot handle high-speed networks.
2.2 Classification by Timing of Information Collection & Analysis
2.2.1 Batch or Interval Oriented IDS
In batch-oriented (also called interval-oriented) approaches, operating-system audit mechanisms or other host-based agents log event information to files and the intrusion detection system periodically analyzes these files for signs of intrusion or misuse.
Advantages:
¢ They are well suited to environments in which threat levels are low and single-attack loss potentials high (e.g., financial institutions).
¢ Batch mode analysis schemes impose less processing load on systems than real-time analysis, especially when collection intervals are short and data volumes are therefore low.
¢ Batch-oriented collection and analysis of information are particularly well suited to organizations in which system and personnel resources are limited.
¢ Attacks on computer systems often involve repetitive attacks on the same targets.
Disadvantages:
¢ Users will seldom see incidents before they are complete.
¢ Aggregation of information may consume more disk storage on the analysis system.
