Cisco Systems Inc. Tuesday claimed that its high-end core router, the Cisco 12416 Internet router, outperformed its competitors in a series of benchmark tests performed by independent benchmarking and network design consultancy Network Test Inc. So why are Network Test and optical networking site Light Reading — which commissioned the tests — crying foul?
Because according to the two firms, Juniper Networks Inc.’s M160 router was actually the best of breed, beating Cisco’s router in three out of four overall areas — IP, MPLS and OC192 (10 GBit/s) — and sharing laurels with Cisco in the fourth category, OC48 (2.5 GBit/s) performance. The two firms said that of the 16 individual tests performed, Juniper’s M160 came out on top in seven, tied for first with Cisco’s router in five, and trailed Cisco’s product in four.
Yet in its initial press release Tuesday evening, Cisco used the headline, “Cisco Reconfirms Leadership in Test of High-End Routers,” followed by “Only Vendor to Demonstrate 100% Line Rate IP and MPLS Performance For 2.5Gbps OC-48 and 10Gbps OC-192 Throughput.”
Light Reading and Network Test, however, maintain, “Cisco did not achieve line-rate throughput in MPLS and IP OC192. In fact, in one of these tests, throughput was just 52 percent.”
Also, contrary to Network Test’s reading that Cisco’s 12416 placed first singly in four of the tests, the release said, “The Cisco 12416 Internet router has outperformed all other networking vendors in nine out of twelve tests.”
Later the company pulled that press release and replaced it with another, changing the headline to “Cisco Reconfirms Leadership in Test of High-End Routers; Demonstrates 100% Line Rate IP and MPLS Forwarding For 2.5Gbps OC-48 and 10Gbps OC-192 Throughput.” And while it continued to claim first place finishes in nine of the 16 tests, it conceded that “Some tests had ties for first.”
Cisco spokesperson Martina Moscone Thursday told InternetNews Radio that Cisco simply chose to interpret Network Test’s results differently than Light Reading.
“We did find the results were very positive and we wanted to talk about them and they just weren’t well represented in the editorial that Light Reading wrote,” Moscone said. “I don’t think it’s just that black and white. If you look, even Light Reading had some difficulty in what they said in their article. When you look back — especially with the MPLS tests — at what the test results show, there’s actually some inaccuracies there. That’s why we felt the need to clarify.”
But in a statement issued by Light Reading Wednesday, David Newman, president of Network Test, said he was disappointed with the way Cisco painted the results.
“I’m very disappointed in this misrepresentation,” Newman said. “Cisco’s 12416 put up some very, very good numbers in this test, so the company had no need to spin it the way they did.”
Newman declined to be interviewed for this article, saying he did not want to further “escalate the war of rhetoric.”
However, Peter Heywood, Light Reading’s co-founder and international editor, told InternetNews Radio Thursday that he too was disappointed in Cisco’s actions.
“This was a very significant test that shed a lot of new light on a real crucial area of next-generation Internet infrastructure,” he said. “A lot of people spent a lot of time and effort on these tests. It took six months and I think a total of $29 million worth of equipment was tested and $2.6 million worth of test equipment was used. A lot of people spent a lot of time on this and really shifted the whole discussion about whose core router is best. We actually established some real facts and presented the results in what we consider to be an impartial way. Those results show that Juniper won and Cisco came a close second.”
Heywood added that the Juniper has been giving Cisco some aggressive complet
ion in the market.
“These results would indicate that that’s likely to continue,” he said. “It must be very worrying for Cisco. I’m sure Cisco went into these tests hoping that they could demonstrate that their latest router was actually going to be better than Juniper’s. I think the results have probably been quite a disappointment for them.”
The tests, performed by Network Test in collaboration with Spirent Communications, were the first comprehensive test of core routers in more than six years, and put routers from Cisco, Juniper, Foundry Networks and Charlotte’s Networks through their paces. Other firms declined to offer their products up for review.
Full results of the tests are available here.
InternetNews Radio Host Brian McWilliams contributed to this story.
