Microsoft’s Visual Studio .NET platform performs
“significantly better” than IBM’s J2EE-based
WebSphere Studio, according to a study released Tuesday by The Middleware Company (TMC), owners of the popular TheServerSide developer communities.
According to the report by TMC, which is a subsidiary of Veritas Software, the goal of the Microsoft-commissioned study, entitled “Comparing Microsoft .NET and IBM WebSphere/J2EE,” was to determine which framework — .NET or WebSphere Studio
Application Developer (WSAD) — made building a business-to-business
enterprise Web application more efficient for programmers.
TMC charged two three-man teams of developers, similarly skilled in their respective frameworks, to build an application that would create and track work orders.
The J2EE team consisted of one member with three years of experience on various Unix
platforms, of which Linux was one of the types; and one with roughly four
years of experience installing, configuring and supporting WebSphere on
multiple platforms, of which Linux was one type. The report did not mention
whether any of the team members had experience with Red Hat’s Enterprise Linux.
Three developers from California-based
Vertigo Software, which specializes in building Microsoft-based
applications, comprised the .NET team. All had between five and 13 years of experience on the
platform.
.NET and WebSphere Studio: A Test
Team J2EE used IBM’s Rational Rapid Developer (RRD) and WebSphere Studio Application Developer (WSAD) running on Red Hat Enterprise Linux AS 2.1 to build two implementations. Team .NET used
Visual Studio .NET running on Windows Server 2003 to create its program. The test was conducted to see which framework fared better in four categories: development productivity, configuration and tuning, performance, reliability and manageability.
The teams were to build an application that created and tracked work orders. From there, the work order was to be processed on a separate corporate network that takes all the order information. A second server would then act like the database while a handheld device would go to someone in a warehouse or shop who has to complete the order, which he physically puts together, accounting for time spent on the order, etc.
While the findings provide developers with a good indication of the benefits of
Windows Server 2003 over Red Hat Enterprise Linux AS 2.1, the study was
ostensibly designed to weigh the merits of developing enterprise
applications using Visual Studio .NET or WebSphere Studio.
The report said the integrated .NET run-time features within
Server 2003 saved Team .NET a lot of headaches because of its built-in Web
server, load balancing and session and message servers.
“The WebSphere team, by comparison, spent a great deal of time during the
development phase installing the software and configuring it for basic
functional tests,” the report said. “They also spent considerable time
overcoming fundamental configuration obstacles, such as patching the Linux
kernel for Edge Server and configuring WebSphere for session replication.
The .NET team did not face such obstacles.”
From the beginning, Team J2EE spent more time on configuration,
installation and bug-tracking than it did on running code using the
development tools. According to the report, the team spent much of its time
“addressing software installation issues and patching the operating system,”
taking 71 days to complete its task. By comparison, it took the .NET
team 24 days to properly configure and test the system.
Also, performance gains went to .NET in the first three throughput tests,
but WSAD achieved “nearly three times the message processing thruput” in the
fourth, the report continued. All three implementations were able to sustain
operations over a 12-hour period under moderate load.”
In some areas, however, it was clear the .NET implementation excelled over
IBM’s tools — WSAD and RDD were not able to handle a catastrophic failover,
while RDD could not add a server to the network cluster after shutdown.
Method or Madness?
An IBM spokesman questioned the results. “The study was paid for by Microsoft, so consider the source” said Steve Eisenstadt. “We think it’s an unrealistic view of the market.”
Microsoft declined comment, saying it preferred questions about the report be directed to The Middleware Company.
Report officials, though, were quick to defend their methodology.
“The Middleware Company has in the past done other business with both
Microsoft and IBM,” the report said. “Moreover, [TMC] is an independently
operating but wholly owned subsidiary of Veritas Software . Veritas and IBM have a number of business relationships in
certain technology areas, and compete directly against each other in other
technology areas. [TMC] stands behind the results of this study and pledges
its impartiality in conducting this study.”
Salil Deshpande, TMC CEO, said he stands by the report. He noted that, while
Microsoft laid the ground rules that .NET compete against
WebSphere, his decision to put the J2EE framework on the Red Hat platform as
opposed to running both on a Windows platform was pertinent.
“We have done several performance studies where we’ve done that, and some of
the feedback we got from those studies was, ‘Why don’t you run the J2EE
platform on Linux, which is where it excels?’ It’s more typical for J2EE shops to go with Linux than deploy on .NET, and it’s also the platform that
IBM is pushing.”
In 2002, IBM signed
a lucrative joint marketing pact with Red Hat to provide Linux-based support.
And in March, IBM announced
it would sell Red Hat Enterprise v3 with its POWER eServer
and BladeCenter lines.
Microsoft, meanwhile, has been actively marketing the cost benefits of its
Windows platform over the Linux operating system for some time, a practice
that’s raised some questions of its own.
Last month, the British-based watchdog agency, the Advertising Standards Authority (ASA), ordered
Microsoft to pull ads about the cost benefits of Windows the group says was
misleading. In January, the company came under fire for marketing its sponsored
research from META Group, Giga Research and IDC.
