An unpublished study from code analysis firm Coverity indicates that
open source database MySQL has relatively few defects in comparison to its
commercial competition.
Coverity analyzed MySQL 4.1.8 last month and found a total of 97
bugs among the open source database’s approximately 425,000 lines of
code. The defects cross over nine different categories, including memory/resource leaks, various forms of nulls and overrun
conditions (including Deadcode, forward null, negative returns, static
over run, resource leak, reverse null, unused value, uninitialized
variable, use after free and unused value).
“Often times in a database we find memory leak problems and resource
leak problems,” Seth Hallem, CEO of Coverity, told internetnews.com. “We
didn’t really see much of that in MySQL, which is a good indication they’ve
tested it thoroughly, and that they’ve designed the code well to avoid those
types of problems.”
Additionally, Coverity’s study found that overrun conditions
comprised only five of the total 97 errors. Buffer overruns have been commonly
used as an attack vector by hackers on a variety of applications, though
according to Hallem, they’re generally not the most common form of
software defect. The Coverity CEO noted that roughly 80 percent of software bugs are null pointer bugs.
“Overruns are perceived as a greater threat because malicious attacks
involve injecting code as a result of overrun, and the effects are more
disastrous,” Hallem explained. “Null pointer may only result in
intermittent downtime and until some hacker figures out how to trigger it
constantly you’d get a denial of service. I think people trigger them
accidentally all the time.”
Coverity’s study did not analyze MySQL on a particular production
platform. In fact it didn’t directly test MySQL on any individual
production platform at all. Instead its analysis looked at the compiled
code base, which in Coverity’s view gives them insight into all production
environments.
“We explore all paths to the code,” Hallem said. “So in essence we’re looking at all and any production environments at all times. And because we’re looking at
all the paths, the production environment may manifest itself as different
pathways or behaviors in the code. We’re trying to look at
all possible behaviors, so in a sense we’re analyzing all production
environments.”
That said, as evidenced by last weeks attack on Windows versions of MySQL
servers, certain environments are more susceptible to attack than others.
“The effects of a security flaw in MySQL and also in many other open
source products are more disastrous on Windows, because Windows does not
have the same kinds of operating system protection available,” Hallem
explained. “That said you’re also putting a lot of trust in the hands of
the administrator. A MySQL admin can always make a mistake and give too
much privilege to a MySQL process on Linux. It’s just that by default it’s
much safer on Linux.”
Though Coverity’s study found 97 flaws, that doesn’t necessarily mean
a hacker had that number of opportunities to compromise or otherwise
attack a MySQL server.
“What it means is that a certain subset of those are actually
triggerable by the outside world. Many of them may be triggered by out-of-memory conditions or configuration issues, or there are many different
possibilities,” Hallem said. “There are some of them and at least one that
we’ve confirmed that could have been triggered by an outside user, and it
was fixed very quickly by MySQL.”
In fact according to Coverity, most of the defects discovered would not be
considered major, though MySQL has now made all necessary fixes. But that
doesn’t mean that MySQL is to be considered 100 percent bug free.
“What it means is that the flaws that our tool can detect have
been fixed, which undoubtedly improves the quality of code,” Hallem said.
“We can’t make any guarantee or any statement as to whether there are any
other bugs that are outside of the scope of what our tool can do. Our
tool does a great job of finding certain kinds of bugs, but we cannot find
everything. ”
According to Zack Urlocker, MySQL vice president of marketing, the results show that MySQL has one-quarter the level of defects or issues compared with
equivalent commercial software. He also noted that a similar result came
from a different study last year, which showed that MySQL had one-sixth the
defect level of equivalent closed source software.
Urlocker said that the
commercial vendors against which the comparisons are made do not publicize
their error rates, so there are no apples-to-apples
comparisons between closed and open source offerings.
“When you can compile your code and not have to show your source code
to anybody, that’s very different than when you are publishing your source
code to your peers and to the whole world, and everyone can see what you’ve
done,” Urlocker told internetnews.com. “That kind of peer review in the
open source world has helped to ensure that the standards by which open
source software is developed are extremely high.”
