An unlikely trio of companies has joined forces to improve and help standardize the way information is crawled or searched on the Web.
today announced support for Sitemaps
0.90 (www.sitemaps.org), a free, unified format for publishers and Webmasters to submit their content.
Google said Sitemap provides an easy way for Webmasters to make their
sites more search engine friendly. It does this by conveniently allowing
Webmasters to list all of their URLs along with optional metadata, such as
the last time the page changed, in order to improve how search engines crawl and
index their Web sites.
A Sitemap is an XML file that can be made available on a Web site and
acts as a marker for search engines to crawl certain pages. The goal is get the information searched on the Web information indexed more comprehensively and efficiently.
“A lot of publishers and Webmasters have been asking for a more
standardized way to interact with search engines, Tim Mayer, senior director
of global Web search at Yahoo, told internetnews.com.
“Now, the content publisher can get their best pages in the index, and it also lets them raise their hands and let the search engine know when there are new pages.
Mayer said standardization will also help with bandwidth efficiency
because the search companies won’t look an extra time at content that hasn’t
Users stand to benefit because the system is meant to get newer results
out quicker. Danny Sullivan, editor-in-chief of Search Engine Watch, lauded
“This is a great development for the whole community and addresses a real
need of Webmasters in a very convenient fashion,” said Sullivan. “I believe
it will lead to greater collaboration in the industry for common standards,
including those based around robots.txt, a file that gives Web crawlers
direction when they visit a Web site.”
In a briefing with internetnews.com yesterday, only
Google and Yahoo were involved, but later in the day, a release was issued
that included Microsoft, a last minute signatory of the deal.
“I am sure this will be the first of many industry initiatives you will
see us working and collaborating on,” said Ken Moss, general manager of
Microsoft’s Windows Live Search, in a statement.
The initiative was originally driven by Yahoo and Google and builds on an
earlier version of Sitemaps released by Google in June, 2005. Mayer said the
companies have “an open invitation” to other search providers, Web
publishers and content management providers to join the standardization
The three companies said they will continue to collaborate on the
Sitemaps protocol and publish enhancements on a jointly maintained Web site which provides all of the
details about the Sitemaps protocol.