Reporter’s Notebook: You could be forgiven for feeling like an unpaid beta tester these days. It seems everywhere we look these days, we’re surrounded by beta software, technologies previews or some other euphemism for unfinished code.
Almost everything on Google except the search engine is stamped BETA. In fact, just recently I was in a local Target and a woman walked by wearing a dark polo shirt, the type so popular with vendors, that had the words “Google GMAIL BETA” stitched in one upper corner.
So, is the shirt in beta, then? Are they testing new threads? New cotton fibers? Or perhaps new stitching styles?
I think you get my point, joking aside. Gone are the days when we didn’t see a product, save for maybe a screen shot in PC Magazine, until it appeared on store shelves.
Can you think of a major product under development at Microsoft right now that is not in some form of public beta? Vista, Office, Internet Explorer, a host of servers, the next Visual Studio, .Net framework 3.0… the list goes on and on.
Java 5 Enterprise Edition and Java 6 Standard Edition were both developed entirely in an open fashion. Every week, Sun posted a new weekly build of SE 6 to a Java community site, while EE 5 was developed as part of Project Glassfish, done entirely in an open source manner.
“What we started with Java SE 6 two years ago was a process to change the culture and attitudes of our engineering team and to create the JDK 6 community development project. We’re making progress and we’re accelerating those changes,” said Jean Elliott, director of developer marketing at Sun.
The granddaddy of them all, or the worst offender, depending on how you look at it, would have to be ICQ. ICQ was one of the first instant messengers, coming out long before AOL Instant Messenger, MSN IM, and so on.
AOL bought the company in 1998 for a hefty $287 million, although since then, AIM, MSN and Yahoo’s IMs have all mowed past ICQ. It’s still around but I don’t know a soul who uses it.
In the four years I did use it, ICQ never left beta. Every version for download was stamped “Beta.” It’s now up to version 5.1 and no longer has “Beta” attached to the version number.
So are well all beta testers now?
Yes, said Jim Duggan, a research VP at Gartner, but that’s to our credit because companies actually think we have something to contribute.
“If you have a sophisticated user set, the user set’s probably more qualified than any of your marketing guys to decide which features should be in the next version of the product. So there are a number of advantages to the vendor of having an open cycle,” he said.
In some ways, it mirrors what the airline industry went through. In the early days of the 707 and 747, Boeing made the plane and said “Here you go” to the airlines and they had to take it or leave it, or at least lobby for changes in future designs.
In later models, particularly the 777, Boeing worked hand-in-hand with the airlines. Everyone from United to British Airways to Cathay Pacific had a say in how the 777 was built from the very beginning of the development process.
Most big firms are enlisting the help of their customer base not because they’ve shipped all their beta testing to India, but because they’ve learned they’re not going to find everything because of the variations in configurations out there, said Duggan.
At 50 million lines of code, Windows Vista is 2.5 times the size of the MVS operating system used on IBM mainframes, said Duggan. So there has to be as widespread testing as possible, to cover all the variations if Microsoft wants to sell it.
“If I said this is all I test against and all I guarantee it works on and ship a product without any kind of anticipation [for variance], it’s going to take a long time to roll up into a lot of enterprises,” said Duggan.
With big vendors, public and open testing has become a necessity. They’re not going to be able to test all the variations, software and platform. And Duggan points out, beta doesn’t mean its bad.
“Beta means they don’t know what’s wrong with it, it doesn’t mean it’s worse than production,” he said. “They don’t have the knowledge to say we guarantee this works well for you, but that doesn’t mean they haven’t done a good job designing it or testing it.”
Indeed, companies have deployed beta software into production settings. Around Christmas of 1999, bookseller Barnes & Noble put Windows 2000, then still in a late beta, to the ultimate test.
It deployed Windows 2000 Advanced Server on its online bookstore, barnesandnoble.com.
Of course, B&N wouldn’t even have considered doing such a thing if it hadn’t spent months testing beta builds of Windows 2000 in advance. Solid betas can be confidence boosters.
Many analysts noticed that Microsoft had a noticeable bump in unearned income for its fourth fiscal quarter ended June 30.
In particular, the jump was in the Information Worker category, which meant Microsoft was racking up a number of pre-orders and license renewals for Microsoft Office. The Office 2007 beta has been very well received and viewed as a solid upgrade.
So, are we winners all around?
We get to try out software before we buy it and get to have a say in its development. Vendors get thousands of free beta testers and a chance to sell the product before it goes on sale. Not much down side, is there?
I wonder if that woman’s shirt is out of beta, though…
