If you haven't seen the news reports, Consumer Reports has been catching quite a bit of flack for their recent test of antivirus products. The methodology they used was to take existing viruses, modify them in some way, and then test to see which virus scanning products picked up their "new" viruses. Apparently many people in the security community think this is a Bad Idea because it involves the creation of "new" viruses. This was irresponsible in their eyes.
I'd like to outline why I think what Consumer Reports did was a good thing and why I am in support of their efforts.
The major virus software developers, including McAfee and Symantec, have an enormously profitable business selling software and virus definition updates. It's a great business idea - people keep paying for the software over and over because it is licensed and not sold outright. The challenge from a security perspective is that antivirus software is more reactive than proactive - it has similarities to the traditional issues with a pattern-based intrusion detection system. They are both great for stopping specific known threats, but do not work as well against unknown threats.
To understand why this is an issue, let's think about how an attacker works. Look no further than 9/11, Richard Reid, or the recent case in London of the liquid bomb plot. The attackers analyzed the security controls in place at the airports and attempted to exploit vulnerabilities in those defenses. They did not try to pack a suitcase with a bomb and check it in because they knew that this was not as likely to work and may be caught in the security scanners. In the same manner, an attacker wishing to distribute a virus can test their new code against the top products in the market just by downloading them. The bad guys are analyzing the defenses to find a hole. It's an endless arms race and the only way to get better is by improving the products to better defend against new attacks.
Focusing on this specific class of product - antivirus - how do you defend from a situation where the bad guys can see your controls and create tactics to evade them? One way is to improve the products in such a way that they are self-defending. McAfee even claims to have done this on their web site: "<VirusScan's> advanced heuristics and generic detection even finds new, unknown viruses."
I'm glad the product is able to do that. As an informed consumer, I'd like to know how well the product stacks up to those claims. As a CISO, it's my business to identify and mitigate risks to my company. I want to know what product can best protect me from both known and unknown threats.
Moving back to Consumer Reports, they figured (rightly so) that the only way to validate the product claims was to modify existing viruses and test them against the software. So they created new variants of known viruses and reported the results. Did their new viruses "escape" and infect anything other than their testbed? To date, nothing has been reported. No damage was done because they appear to have employed care in how they conducted the test. I would expect this from an industry leading product evaluation company that brought in competent security consultants such as Dr. Avi Rubin.
What's the bottom line? Consumer Reports obtained an independent test result that I am very interested in - which products were best able to cope with new and evolving threats. This information is valuable to me because it was created by a credible not-for-profit institution and provides details to help me choose the best product for defending against both existing and new threats.
[