By Joel Snyder
Network World, October 15, 2007
Original Article on Network World Web Site
Before committing to any messaging-security gateway, it's critical to test the product in your own network e-mail stream.
While some features, such as system performance and content filtering, can be tested off-line, there is no way to tell how good an antispam and antivirus gateway is until you see it operate on your e-mail stream.
There's no substituting some kind of reduced-fat cheese product for real cheese here. You can't make a copy of e-mail; you can't send messages via an alternate route; you can't send through stored e-mail. You have to put the box in the mail stream, in real time, to see how it responds to spam and viruses.
The bottom line is that the antispam gateway has to see the mail as it comes in, from the spammers, in order to extract the most information about the mail and provide the appropriate response to the attackers. If you are using security settings, such as encryption requirements or IP-based content filtering, you'll also need to have that device in its final position to test correct operation.
To help ameliorate concerns about putting new gear into a production network, we often suggest to our clients that they keep their old messaging-security gateway "inside" of the new test system. In other words, mail comes to the new system, then the old system, before it passes to the e-mail servers.
This has some nice benefits: You can easily see if the new system is letting viruses or spam through that the old system would have caught by looking at the logs and counters on the old system. You can't really tell how much better the new system is than the old at catching spam and viruses, although some testers have tried a shift approach: swapping boxes every few hours (you want to have round-the-clock coverage, because spam and virus patterns vary by day of week and hour of day) and comparing statistics between the two.
Another, albeit counterintuitive, testing guideline is to make sure the product vendor doesn't get to tinker with the antispam part of the device. A good antispam product should work out of the box without substantial tuning and adjustment. A few tweaks are OK, but if your sales engineer is spending most of a day on "tuning," then you've got a long-term problem: Your box will slowly go out of adjustment as both spam patterns and your e-mail change. A messaging security gateway is not like a car: It doesn't require regular tune-ups to operate at peak efficiency, at least not if it was built right in the first place.
Of course, that warning doesn't apply to other parts of the system, such as content filters or encryption and privacy settings, all of which can take a while to get right through a process of trial and error.
Test it like you mean it. Our observation is that by the time a vendor gets to the stage where they do an on-site test installation, the odds are good that the product is going to work. This means that your test installation can turn into permanent production very quickly.
To smooth this process, make sure you do the test installation with the same care and processes you'd give a final installation: Get the box racked, powered, cabled and labeled properly; make sure user names and access controls are what you'd select for a production system; set defaults, reporting, alerting and configuration options as if this were really going to handle all your mail. If you're buying enough product, your vendor will probably be sending a sales engineer to help on the installation, and no one is going to be able to get that system in more professionally and with the same attention to detail.
This has three benefits: First, you'll discover hidden gotchas that you wouldn't have found until too late. Secondly, you'll have the benefit of a top-notch engineer on-site for the "final installation." Third, you'll save time if you decide to keep the systems by not having to redo the work.