The Open Source Fight Club
Page 1 of 1
SAN FRANCISCO -- The first rule of Fight Club is to not speak about Fight Club. That is unless of course you're an open source vendor that, by definition, aims to make everything open and available.
Open source networking vendor Untangle wants to talk about its Fight Club, an event, scheduled for LinuxWorld here this week, that will pit open source antivirus and gateway security solutions against their proprietary competitors. The event could highlight the validity of open source solutions in a space that Untangle alleges has been held back in a conspiracy by testing labs.
"We believe test labs get paid by proprietary vendors and have no motivation to show a free and open source solution is better," Dirk Morris, founder and CTO of Untangle, said. "Because of their lack of transparency, we are left to assume they are not performing fair tests or aren't testing at all because they fear the results."
Morris explained that two years ago, Untangle decided to add antivirus scanning to its network gateway solution. At the time they did a "bake off to see which vendors are best from a technology fit and performance angle.
Morris figured that antivirus technology was fairly commodity and that everyone would score fairly similarly, but it turns out his initial assumption was wrong.
The methodology used by Morris was straightforward. He pulled a set of viruses off the Web and out of his e-mail. None of the viruses were less than two months old. His testing found that most vendors caught less than 50 percent of the viruses, and in general the whole group did very poorly.
"To our surprise, ClamAV [open source antivirus], which we thought sucked at the time, caught the most viruses -- all but one -- and did so using the least amount of computing resources," Morris said.
Untangle then integrated ClamAV into its solution and approached testing labs to get various security certifications. According to Morris, some refused to test the Untangle solution because of its use of ClamAV. At the same time, Morris said that the testing labs claimed they had tested ClamAV and it had done poorly and refused to give Untangle any details, results or test sets.
Though the Fight Club event at LinuxWorld is a short test in terms of time duration, Morris argues that it will still prove his point.
"We aren't testing any bleeding-edge viruses and only viruses found in the wild. Every vendor in the test should catch every virus," Morris said. "However, I think you'll see that some solutions are distinctly poor despite public claims, and that open source is one of the best, if not the best, alternative."
Untangle isn't the only open source networking vendor that will be at LinuxWorld, open source networking vendor Vyatta will be there, too. Though Vyatta also makes use of ClamAV in its solution, Dave Roberts, vice president of strategy at Vyatta, told internetnews.com that he might drop by for grins, but they aren't participating.
Roberts also does not see the same conspiracy against open source that Untangle's Morris sees.
"The fact is testing houses get paid to do testing. As a result, they work with products where somebody can step up to fund that testing," Roberts explained.
"If an open-source project isn't well organized and funded, the testers won't spend the time on a project that isn't going to bring some income. Many open source projects are a loose affiliation of people with little organization structure. In some portions of the development model, that's a great strength; in the case of funding testing, it's a weakness."
Roberts admitted that there is a lot of open source that is under-tested. That's one reason why Vyatta is a big supporter of the commercial open source model since commercial entities have the funding and resources to sponsor deep testing of code.
"At Vyatta, we have sponsored testing of our code base at the University of New Hampshire's Interoperability Lab (UNH-IOL), and we published all the results on our Web site," Robert noted. "If the Untangle event results in better testing for open source AV solutions, then the community wins, and that's a good thing for everybody.
Though it's not participating in Untangle's Fight Club, Vyatta has its own event in which it is hoping to help out open source.
The Vyatta Open Arcade Challenge will have bunch of classic arcade games -- Asteroids, Pac Man, Defender, Donkey Kong, and Centipede -- for folks to play; Roberts said all the proceeds from the event will go to various open source projects.
In either case both events certainly sound a lot less violent than Brad Pitt's Fight Club.