MCI WorldCom Inc. Sunday finally restored its domestic frame relay network platform in the U.S., following a planned 24-hour outage in which it sought to correct problems that have gone on for 10 days.
In order to stabilize their frame relay network, MCI WorldCom Saturday took the domestic system out of service for approximately one day. During this period the frame relay service was not available to anyone. As a result, customers of Internet service providers who use MCI as their as their backbone carrier were unable to surf the Internet.
America Online Inc. notified members that they anticipated 10 percent of their 17 million subscribers would experience problems connecting during the MCI scheduled maintenance. In many of the areas of the U.S. where AOL offers local dial-up access, alternative upstream connections were used to provide uninterrupted service to the Internet.
However, 50 cities throughout 28 states that subscribed to AOL did not have alternative local access numbers to utilize during the UUNET outage. AOL provided these members with a 1-800 number to dial-up free of charge, so the majority of customers impacted by the MCI outage were minimized.
Bernard J. Ebbers, MCI WorldCom president and chief executive officer was apologetic for the difficulties incurred by the network outage, but did not say whether MCI planned to compensate its customers for the downtime.
“We have resumed providing service to the customers who were impacted and will be working hard to regain their full confidence. MCI WorldCom is very committed to providing the highest levels of network reliability on all of its services to all of its customers,” Ebbers said.
Ebbers added that the outage has been an equally challenging time for both the company and their customers. He said that consumer confidence was tantamount to restoring normal network operations.
“Every MCI employee has been and will continue to be working very hare to restore the confidence of our customers,” Ebbers said. “We’ve completed the first step to restore service, now we plan to offer all of our customers a 2-day credit for free service, for each day the network was out of service.”
Approximately 3,000 customers may be asking for more than a double-day credit plan from MCI WorldCom. Although the total financial impact has yet to be tallied by local and regional ISPs, or companies like AOL and the Chicago Board of Trade that were impacted by the service outage.
Lucia Graziano, Lucent Technologies director of public relations, said that Lucent technicians worked around the clock with MCI World/Com to stabilize the network.
“We’ve learned a great deal about identifying, isolating an repairing network degradation over the past few days,” Graziano said. “We are still working to investigate the root cause of the problem. All we know for certain is that the problem occurred at the point of installation.”
Ron Beaumont, president of network services, said that the company is still investigating some of the software issues that impacted their network status.
“We do not have definitive cause of the problem. Lucent has acknowledged full responsibility for the problem and we know it is a software problem.”
Beaumont said that MCI WorldCom first detected system degradation though standard network monitoring on Thursday, August 5th. However, the software upgrade from Lucent started over 4 weeks ago. The new software was designed to make the frame relay network scalable in order to better handle peak access and load balance the system.
Beaumont estimates they their current frame relay network can sustain current demands for another 18-months before needing to either expand the network, or upgrade the switching software. Beaumont guaranteed that future upgrade would work better in the future.
“Going forward we will work even more closely with our vendors on preliminary testing before installing any substantial upgrades,” Beaumont added that “we’ll do everything we can to win back our customers confidence.”
Ebbers said that no customers have informed MCI WorldCom that they intend to leave their services as a result of the upgrade and that the outage will not produce longer-term financial issues for the company.
Linda Laughlin, MCI WorldCom manager of media relations, said the company is continuing to speak with ISPs about their concerns.
“We’re continuing to talk to access providers, one-on-one in order to determine how they were impacted by the frame network recovery and any type of compensation that may be required.”
Problems in the network were traced to a glitch in software provided by telecommunications equipment giant Lucent Technologies Inc.
The outages affected about 15 percent of MCI WorldCom’s network and 30 percent of its customers who rely on high-speed frame relay networks to exchange large amounts of computer information in short and frequent bursts.
Laughlin said that the MCI WorldCom frame relay network is back to its original form and that the software upgrade is back in testing for the time being.
“The network degradation occurred when new software was installed. We’ve reconfigured the programming, removed the software and stabilized the network to provide the reliable service MCI WorldCom / UUNET is known for.” Laughlin said. “The software is back in the labs for testing and we have no new timetable for future frame relay network upgrades at this time.”
