RealTime IT News

Tackling the RSS Bandwidth Bugbear

The growth of RSS usage as a platform to syndicate content to feed readers and aggregators has led to a dramatic increase in bandwidth consumption for publishers.

Bloglines founder Mark Fletcher believes his six-employee firm has come up with a solution to the bandwidth bugbear. Employees at the server-side RSS aggregator have released a set of Web Services APIs in a bid to reduce the bandwidth burden faced by content publishers.

Called Bloglines Web Services (BWS), the beta release allows software developers direct access to Bloglines' subscription and feed data directly from third party programs.

"Basically, the aggregators will be sending queries to the Bloglines database instead of pinging servers owned by individual publishers," Fletcher said in an interview with internetnews.com.

The bandwidth problem came to the fore earlier this month when Microsoft turned off full-text feeds from its MSDN portal because it was having problems keeping up with the constant hits on its site for feed updates from RSS aggregators. At the time, Microsoft software evangelist Robert Scoble wrote on his Weblog that bandwidth usage was becoming a financial burden on the software giant.

"RSS is broken," Scoble wrote, adding that it was not scalable when tens of thousands of people start subscribing to thousands of separate RSS feeds and start pulling down those feeds every few minutes. "Bandwidth usage was growing faster than MSDN's ability to pay for, or keep up with, the bandwidth. [Terabytes] of bandwidth were being used up by RSS," he added.

MSDN would later streamline the way the full-text feeds were being distributed but the issue stirred a debate in the blogosphere and prompted Bloglines to search for an answer.

The answer, Fletcher believes, can be found in the Bloglines APIs, which enable Bloglines to act as a feed cache as a way to insulate content providers from having to deal with high bandwidth consumption. "The individual publisher wanting to put out an RSS feed won't have to deal with the technical ins-and-outs of setting up a massive server cluster to handle the bandwidth issue. The APIs alleviate that problem," he said.

Fletcher, who launched Bloglines two years ago after selling eGroups to Yahoo , said the Web Services API will spit out data as well-formed RSS 2.0 or Outline Processor Markup Language (OPML), a W3C recommendation for exchanging lists of RSS feeds between RSS aggregators. The bottom line is that developers won't have to worry about having to support multiple syndication formats.

He said desktop aggregator developers can use the API calls in order to sync aggregators across multiple computers while giving users access to a Web-based aggregator.

Just 24 hours after releasing the Bloglines Web Services, Fletcher said a developer had used it to create a PERL module.

He also said desktop aggregators like FeedDemon, NetNewsWire, and Blogbot all plan to add support for the APIs so that users can store their RSS feeds on Bloglines and read those RSS feeds from any browser.

Fletcher's move comes just a day after Yahoo rolled out a new version of its My Yahoo service with plans to introduce RSS to mainstream users.

At the same time, as RSS usage grows, players are scrambling to build partnerships in the budding sector. For example, NewsGator, a Bloglines competitor, just announced three partnerships aimed at driving adoption of RSS as a content syndication platform.

NewsGator founder Greg Reinacker told internetnews.com the company's deal with FeedDemon will let them to connect to subscribers of NewsGator's Online Services. "FeedDemon users will now be able to read their selected content not only on their desktop, but also on the web, on their mobile devices, on their Media Centers, or in Microsoft Outlook," he said.

Reinacker also announced a separate deal to integrate news search data from Moreover to NewsGator's search feeds. The partnership also calls for NewsGator to carry Moreover's aggregated news search feeds in its desktop and online products, providing topic-specific information to users.

A third partnership with blog tools vendor Six Apart sets up a co-marketing relationship to push RSS and blogging to enterprise users. Reinacker said NewsGator is working on new plug-ins for Outlook to make it simple to post feed content from NewsGator products to Six Apart blogs in Movable Type and TypePad.