Homeland Security Helps Reduce Open Source Flaws
Page 1 of 2
Nearly two and a half years ago, the US Department of Homeland Security (DHS) issued a multi-year grant to help improve open source code quality. It appears that the DHS investment has paid off.
According to a report from code analysis vendor Coverity, the DHS sponsored effort has helped to reduce the defect density in 250 open source projects by 16 percent over the past two years. That defect reduction translates into the elimination of over 8,500 defects. The report on the benefits of the DHS open source security efforts comes at a time when open source software is increasingly becoming part of critical infrastructure both in the government and in US enterprises.
"The improvement of project defect density is such that when we started the effort they were at 0.30 defects per thousand lines of code and now they are down to on average 0.25 defects per thousand lines of code," David Maxwell, open source strategist for Coverity told InternetNews.com. "I know that feels like a small percentage change but when it's over 55 million code it adds up."
Coverity is a code analysis vendor and runs its scanning tools on the included open source projects to identify coding errors.
While many projects have benefited from running the DHS sponsored Coverity scan, not all have actually managed to reduce their defects.
"There is a graph in the report that shows some project have significant improvements and some that haven't been actively using the results from the scan that have actually increased in defect density," Maxwell commented.
The report graph that was provided to InternetNews.com doesn't fully name the names of those project that did not improve as a result their defect densities. The report however did identify Perl, PHP, Python, Postfix, Samba and TCL among the projects that have been able to reduce their code defect densities by using data from the Coverity scans.
Coverity's scanning efforts have also provided them with some interesting statistical data points about which type of errors seem to occur more often than others. Leading the pack are Null Pointer Dereferences at nearly 28 percent.
"This type of error often occurs when one code path initializes a pointer before its use, but another code path bypasses the initialization process," Coverity said in its report. "Because pointers are often used to pass data structures by reference between pieces of program logic, they may be the most commonly manipulated data objects due to repeated copying, aliasing and accessing. Therefore, it is not surprising that the most frequently used artifacts will incur the most errors in manipulation."
Coming in second for recurring defects are resource leaks at 26 percent of all defects.
"Resource leaks often involve failure to release resources when the initial allocation succeeds, but a subsequent additional required resource is not available," the Coverity report explains.
Maxwell commented that in looking at the volume of data they were able to collect on open source projects there were some things that went against their expectations.
Among the findings that Coverity did not expect is one that has to do with function length and its relationship to defect density. Maxwell explained that there is a common myth or misconception that if a code function is too long and for example can't fit on a programmers screen, then it is more defect prone. As it turns out, that's not the case.
"The defect density did not increase," Maxwell said. "There is almost no correlation between average function length in a project and a project's defect density."