The pillar of most open source applications is the ubiquitous LAMP — Linux,
Apache, MySQL, and Perl/PHP/Python stack. According to results
of a Coverity study, the LAMP stack has
fewer code defects than a baseline of 32 open source software projects.
The
effort may well prove to help make open source more secure.
The Department of Homeland Security (DHS) announced the project, called “Vulnerability Discovery and Remediation Open Source Hardening Project,” at the beginning of the year. It includes a grant from DHS of $1.24 million to be dispersed over three years
to Coverity, Stanford University and Symantec.
The first results of the Coverity scan report that the average defect
density in LAMP code is currently 0.290 defects per thousand lines of code.
A baseline scan run by Coverity of 32 popular open source projects, which
also includes the five LAMP applications, found an average defect density of
0.434.
Previous Coverity studies (not backed by DHS) have found low defect
densities in the Linux kernel and in the MySQL database.
Ben Chelf, Coverity’s CTO, commented that he wasn’t surprised with the defect
distribution shown by the new analysis.
“The average defect density was probably about where we thought it would be,”
Chelf told internetnews.com.
Chelf was surprised, however, by the performance of the Perl language.
“Of the LAMP stack, Perl had the best defect density well passed standard
deviation and better than the average, Chelf said.
Perl had a defect density of only 0.186. In comparison Python had a defect
density of 0.372 and PHP was actually above both the baseline and LAMP
averages at 0.474.
Other popular projects included in the 32 projects that made up the baseline
are: Firefox (0.355), Gaim (0.352), Gnome (0.458), GCC (0.202) and Samba
(0.695), among others.
Apparently the results also show that larger projects don’t necessarily have
more or fewer defects than small ones.
According to Chelf, “there wasn’t a correlation between size of code base
and defect density.”
The goal of the DHS-backed effort isn’t just to identify bug defects, but
also is intended to help make open source applications more secure, as well.
“The thing about security vulnerabilities is that any defect in code can be a
security vulnerability,” Chelf explained. “It’s all about how that defect is
triggered.”
A crash condition, for example, could be considered a vulnerability if a user
has the ability to poke at it and trigger the crash.
Chelf also added that
the scan detects specific defect classes that are tagged as security
vulnerabilities. Those defects have to do with the way an application
handles tainted data coming in from the outside, essentially making sure that
an application does the “right thing” with data that is input via a
keyboard or over a packet.
Identifying issues also isn’t the only goal of the effort; according to
Chelf the identified defects are actionable.
“Measuring code is good. Fixing code is better,” Chelf said. “I think that’s
the big next step, working with the maintainers of these projects and saying
‘hey this data is available and let’s talk about how to fix the problems.'”