Commercial vendors sometimes take pot-shots at open-source software projects
because they think the code is weaker, or hasn’t been validated by
professional testers. But a new study by an inspection firm has found that the
latest version of Apache Web server is comparable in quality to its
commercial brethren.
Mountain View, Calif.’s Reasoning, which charts the degree of flaws in such
programming languages as Java, C++ and C, found 31 software defects in
58,944 lines of Apache Web server V 2.1 code.
Using the industry-accepted reliability indicator called defect density,
which is the number of defects found per thousand lines of source code, the
group found a defect density of the Apache code of 0.53 per thousand lines
of source code. Meanwhile, the average defect density of commercial code was
0.51 per thousand lines of source code.
Measuring those findings against the findings in a similar test from February,
which found that the TCP/IP protocol stack implementation in version 2.4.19
of the open source Linux kernel has fewer defects than the TCP/IP protocol
stacks of several commercial equivalents, the testing company concluded that
maturing open-source software can be as high in quality as commercial
vendors’ software products.
The findings could leave folks to draw their own conclusions, but for those
advocates of commercial products, it could poke holes into their oft-floated
assertions that software that is sold is of higher quality — an argument
leading vendors have made when trying to discredit open-source operating
systems like Linux, or open-source databases such as MySQL.
Bill Payne, President & CEO of Reasoning, said that February study, which
concluded that open source had a significantly lower defect density compared
to commercial equivalents, led for developers to call for another similar
test, albeit one in which the open-source application was less mature.
“We received numerous inquiries about that study and took seriously requests
for us to examine defect density rates in a less mature Open Source
application and compare it with the commercial equivalent,” Payne said.
“Taking advantage of our database of automated software code inspection
projects, we were able to do exactly that, and found the difference in
defect density between the two was not significant.”
Reasoning’s inspection service is based on a combination of proprietary
technology and repeatable process. The company pledges objective results
that are comparable across software applications, development methodologies,
and coding styles.
The results of Reasoning’s Apache 2.1 inspection can be obtained free here.