Content is king

Attack signatures trigger a range of responses among content-based IPSs.

By Joel Snyder, David Newman and Rodney Thayer,
Network World Global Test Alliance, Network World, 02/16/04

Original Article on Network World Web Site

The in-line products we tested were Check Point's InterSpect, EcoNet.com's Sentinel IPS, Internet Security Systems Proventia G Series, Lucid Security's ipAngel, NetScreen Technologies' NetScreen-IDP 100, StillSecure's Border Guard and TippingPoint Technologies' UnityOne. Because EcoNet.com is a managed service rather than a stand-alone product, we discuss it separately (see Managed IPS alternative).

We installed each of these in our labs in Los Angeles, San Jose and Tucson, Ariz., (see How we did it) and assessed them from the perspective of network professionals looking to put an IPS into a production network.

• What does the product catch? What kind of malicious traffic is this designed to identify? Where did the engineers design this product to go in a network?

• How does the IPS block traffic? What other reactive techniques are available?

• How can the IPS be controlled? What features are available for management, configuration and tuning?

ISS, NetScreen and TippingPoint clearly fit our model of how an enterprise product should be built.

All six had some level of signature-based intrusion detection to help identify malicious or anomalous traffic. After that, we found four with limited rate-based control capabilities, two with connection flood (also called SYN flood) controls and one with built-in honeypot technology.

Finding intrusion-detection system (IDS)-style signatures and protocol-anomaly detection in these IPS devices was no surprise. IDS vendors are ideally situated to design IPS products because they've already thought about what it takes to identify malicious traffic. In three cases, the IDS inside looked very familiar. IpAngel and Border Guard are built on top of the open source Snort IDS engine. Proventia uses the ISS IDS engine inside.

Proventia ships with the entire ISS signature library, but only about 250 rules are enabled by default for the IPS function. These are rules that ISS is willing to guarantee will not generate false positives. We found a similarly reduced list in InterSpect and UnityOne. Balancing a short signature list to reduce false positives with enough signatures to make IPS useful is a constant battle for vendors as these products are installed and updated.

NetScreen has a huge signature library, but you have to define your internal hosts and vulnerable ports for the signatures to apply. For a large network, that would be a fairly tedious process. NetScreen will add automation tools in the next version of its IDP, shipping this quarter.

In a unique tack on turning signatures on and off, Lucid Security configures its ipAngel detection engine based on feedback from a vulnerability scan from a Nessus open source network scanner. If the scanner finds something vulnerable, ipAngel enables the IPS/IDS signature. Otherwise, it's turned off.

Border Guard and UnityOne use a built-in nmap vulnerability scanner, but neither are as sophisticated in their use of scan data as Lucid is. Strangely enough, ISS, which sells one of the top vulnerability scanner products, has not yet linked its vulnerability scanner and IPS products.

We also found honeypot technology in NetScreen's IDP. The idea behind a honeypot is that most attackers will do very broad-scale reconnaissance on a network as part of an attack. If you put a system out there that should never be legitimately connected to, then any connection to that honeypot system is suspect and represents potential malicious traffic, no matter the content. IDP can use specifically configured honeypot addresses and services to initiate a block against further traffic from the system connecting to it.

Rate-based controls were a welcome feature in these content-oriented IPS products, even if they did not meet the sophistication of other rate-based IPSs we looked at. Check Point, ISS, NetScreen and TippingPoint all brought rate-based controls to the table.

Check Point and NetScreen included sophisticated protection for connection floods with a TCP proxy. For example, NetScreen's SYN Protector feature lets you define a combination of IP addresses and an application, then enable the protector. All TCP connections are proxied by the SYN Protector, eliminating some classes of connection flood attacks. The content-based IPSs we tested don't have any sophisticated tools for User Datagram Protocol (UDP)-based protocols.

UnityOne, with its traffic management features, best straddles the line between the rate-based and content-based IPS camps. While it doesn't offer comparable intrusion-protection power of the best rate-based products we tested, it does offer detailed bandwidth controls (source and destination addresses and application), and signatures that detect high connection rates.

What does it do?

We found that once bad traffic is identified, the IPSs we tested can:

• Drop the malicious traffic.

• Drop all future traffic on the same TCP or UDP connection.

• Actively try to close the connection by sending TCP reset packets to the client and server.

• Aggressively drop future traffic related to the attack traffic (for some period of time), such as from the same source IP address or network.

We expected that any IPS always would drop a malicious packet. We were surprised to find that ipAngel and Border Guard don't always. Both detect problems within traffic, but use that information to modify the behavior of an associated firewall running on the IPS device, dropping future traffic from the offending IP addresses for some period of time. Lucid uses a Check Point Firewall-1; a proprietary firewall is included in the StillSecure IPS.

StillSecure also has a pre-emptive mode that uses compiled Snort signatures to drop traffic before it can pass through the IPS. The problem is that Snort is more powerful than StillSecure's firewall and will catch some traffic that the firewall will pass. This is especially true in cases where an attacker intentionally tries to evade the IPS or obfuscate the underlying datastream.

All the IPS products, except for UnityOne, had the option to create dynamic, short-lived blacklists designed to protect the network from attackers. TippingPoint offered the option of limiting the bandwidth of types of malicious traffic. For example, if you want to allow pings, but not ping floods, you can write a signature to match ping (Internet Control Messaging Protocol request) packets and then permit, but rate-limit, them. ISS has a similar feature. We didn't look directly at the issue of writing signatures, although ISS, NetScreen, StillSecure and TippingPoint all let you define your own.

There was little consensus among IPS vendors when we looked at how they addressed dropping active connections and blocking future traffic. For example, ipAngel not only drops the offending packet, but also all future packets from the attacking system for the next 60 seconds.

That simple blacklist strategy contrasted sharply with ISS' approach. For each signature, ISS lets you define a variety of reactions, including simply dropping packets, closing connections or updating a blacklist. If you do update a blacklist, it's not just a "drop everything from the attacker" choice. ISS lets you define many different blacklisting strategies. The defaults let you block future traffic from the same attacker to the same victim or combine IP addresses and applications. The simpler option isn't even available. In a very obvious way, Lucid (and other vendors, including Check Point and StillSecure) disagree completely with ISS in their blacklisting strategy.

It's hard to say which is the "right" way to handle bad traffic, but the conservative approach ISS offered seemed like it would get you in a lot less trouble with self-inflicted denial-of-service attacks over the long run.

In addition to dropping malicious traffic and adding IP addresses to blacklists, some IPSs give you additional options, ranging from dropping all other traffic for that particular connection to actively trying to tell the client and server that the connection is closed by sending TCP RST segments in both directions.

The problem is that not every signature deserves the same reaction. For example, a TCP packet with a wildly wrong sequence number probably shouldn't cause a connection to be broken and a blacklist entry to be made, because it might be a forged packet. If you let anyone send you random TCP packets to shut down other people's connections, you'd have a particularly brittle network. We appreciated the work that ISS and NetScreen did in designing the appropriate reaction to every signature rather than treating the entire IPS as a monolithic entity.

We also were concerned about how these products behaved when they encountered bad traffic: what information was kept and how could the network manager use it. Products took two tacks on this. ISS' and TippingPoint's products behaved like an IDS, providing a comprehensive forensics capability and detailed information about what happened and when.

Check Point, NetScreen and StillSecure took a more traditional reporting strategy, aggregating and collecting data. Check Point stands out with a tremendous set of reporting and logging tools. Because InterSpect is closely derived from Check Point's Firewall-1, all the tools that are part of Firewall-1 are available in InterSpect. Having all that power actually makes the product look lopsided: the reporting side of InterSpect, with nearly 10 years' worth of development and experience behind it, is more mature and complete than the newly written IPS side.

How can I control it?

The scariest two words to an IPS vendor are "false positive." These folks want you to trust the most critical parts of your network infrastructure to them. While some, such as Lucid Security and StillSecure, have a posture clearly aimed at protecting you from the Internet, encouraging you to place their devices at the perimeter of the network, the rest want you to put their boxes deep within your network. At that location, they can't risk false positives - it's better for bad packets to get through than good packets to be blocked.

One of the first management features we looked for was the ability to put the system into alert-only mode. The idea is to keep the IPS running, but never drop any traffic. You would want to do this for tuning purposes, and a network professional might want to run it in this mode if the IPS is ever suspected of causing network problems. ISS understands this issue and gave us a nice big button in the GUI to put its Proventia into alert-only mode. NetScreen pointed to its configuration versioning capability, which would let you create two configurations, one alert-only and one not, along with the ability to easily switch between them. All the other IPSs had a hard time with this simple request, either requiring some hardware rewiring or a more detailed modification of the security policy that was not easily reversible.

We also thought that most network professionals would want to have a whitelist capability: Tell the IPS that certain systems are not to be blocked for any reason. ISS, NetScreen, StillSecure and TippingPoint gave us nice levels of detail, down to the port or, even to the signature level. Check Point's whitelist function looked good in theory, but because of bugs in the late beta version we tested, we kept losing systems we added. Lucid had a less granular whitelist, which would probably be reasonable for most networks.

Another customization issue was network discovery. For many application-layer signatures, there is an implicit assumption that particular applications run on particular ports. We wanted to see how the IPS devices adapted to our networks, including applications running where they didn't belong.

IpAngel looked, at first glance, like the answer to our problems. The built-in Nessus scanner activating rules seemed like a great solution. Scan your network once in a while, turn on and off the appropriate rules, and you're all set. But in our tests, Nessus might have found our non-standard mail server on Port 2525, but ipAngel didn't activate any signatures for that port. With ipAngel's very weak Web-based GUI, we didn't have the option to fix this deficit ourselves.

We had a similar problem with UnityOne, which uses the simpler nmap tool for system discovery. You can't touch the configuration after nmap defines what the ports on which particular protocols run. StillSecure has nmap, but this feature is not fully fleshed out. You can use the results of an nmap scan to block traffic to nodes which nmap finds, letting through traffic to systems that don't exist.

On the other hand, products from ISS and NetScreen don't have automated discovery tools. Check Point's InterSpect doesn't give you the option of defining services - if you're running an application on a non-standard port, you don't get to protect that protocol with their application-specific IPS features.

Wrapping up

One of the most solid products was UnityOne. With a clear interest in core-of-the-network implementation, it offers a good base for a simple IPS. TippingPoint didn't stand out with flashy features, but the architecture of the product and the capabilities it did offer make it a product to watch.

In the category of products we'd buy for our own networks are the ISS and NetScreen boxes. NetScreen's clean implementation looked solid in every way. ISS, likewise, clearly brings a serious understanding of what an IPS should do to this market.

Captus IPS 4100XT 1.2
Company: Captus Networks, (877) 922-7887 Price: $12,000 for appliance; $10,000 for management console. Pro: Advanced policy very flexible for complex environments. Cons: Management system doesnít match product well; doesnít offer network performance data to help tune; canít limit TCP connections.
Sleuth9 3.5
Company: DeepNines Technologies, (214) 273-6996 Price: $25,000 for unlimited users and includes Holistic Management Console. Pros: Includes virus scan; offers many graphical reports on network state. Cons: Underlying operating system hardening inconsistent; weak performance data; some IPS features can't be turned off; weak documentation and help; no reporting.
Attack Mitigator IPS 100 2.1.016
Company: TopLayer Networks, (508) 870-1300 Price: $15,000 Pros: Good combination of rate-based with some content-based features; mirror port good; application connect rate and count blocking excellent; good performance monitoring. Cons: Little reporting; no centralized management option means data can be lost if buffers overflow.
NetProtect Enterprise LG100 LAN Gateway
Company: Vsecure Technologies, (888) 895-7500 Price: $20,000 to $25,000 Pros: Good whitelist design; innovative configuration model well thought out. Cons: Confusing configuration system; protected hosts require comprehensive service description because of built-in firewall; poor documentation; no rate limiting, only blocking for malicious traffic.