Search Leaders, Bloggers Band to Fight Comment Spam

MSN, Yahoo and blogging software vendor Six Apart said they would adopt a
tag
proposed by Google and join the effort to fight the comment spam that can skew
search results.

Most search engines use popularity as one way of evaluating a site’s
worth. If lots of other sites link to it, the reasoning goes, it must
have valuable content. This was the insight that made Google
a search leader, and it’s now used to some extent by the other major search
players.

However, with the rise of search engine optimization and affiliate
marketing,
some people began to game the system, trying to insert links to their own
sites
wherever they could. The comment sections of blogs were a top choice for
posting
comments like, “Great site! Check out mine,” along with a link to the
writer’s
own site.

Google’s proposed tag, (rel=”nofollow”) tells search engine spiders to ignore the link
when
indexing.

On the Google corporate blog, Matt Cutts, a Google software engineer, and
Jason Shellen, Blogger program manager, wrote, “From now on, when Google
sees
the attribute (rel=”nofollow”) on hyperlinks, those links won’t get any
credit when
we rank Web sites in our search results.”

“The problem they’re addressing is comment spam,” said Kevin Lee,
president
of Did-It, a search engine marketing company. “People will write robots that
clutter
up your comments area with hotlinks that strip your page rank and shoot it
off toward
the spam site. Google’s spider sees the links and says, ‘This blog thinks
that site is good.'”

Lee said spammers can potentially add their own links to tens of
thousands of blogs.
“It gives them an amazing advantage in spamming [search engines],” he said.

It’s notable, said Danny Sullivan, editor of SearchEngineWatch,
that Google
proposed an industry standard and was joined by its rivals in making the
announcement.
(SearchEngineWatch and internetnews.com both are owned by
Jupitermedia.)

“They had already decided to do it and talked to some vendors about the
idea,”
Sullivan said. Google engineers evidently contacted MSN and Yahoo at the
beginning of
the week.

“One thing that helped is that both companies were already considering
ideas
like this. Google had a solution that looked like it would work and some
vendors who were
already on board with it. That shouldn’t diminish the fact that they did come out in a unified
voice,” Sullivan
added.

The last time that happened, he said, was in 1996, when a spidering
work group
composed of employees of the different search engines came together to
decide on a meta robot
standard to block spidering.

Jeremy Zawodny, a Yahoo marketing executive, said “Every company in this
business has
been thinking about this, and some of us have been writing about possible
ways to combat
comment spam. We had to start with something, and we started with this.”

He said that since Yahoo doesn’t host blogs itself, the use of the
no-follow tag would
take time to make its presence felt in Yahoo’s index.

“It’s not a difficult thing to do,” he said, “but it’s not like flipping
a switch. It’s
a question of how long it takes for those changes to propagate through our
index. People
should start to notice the effects in the next few weeks.”

MSN product manager Justin Osmer pointed out that MSN Spaces already lets
bloggers limit
access, require authentication before posting comments and delete comments.
He said the company’s
“Search Champs,” a group of search industry advisors, had suggested
something similar.

“MSN supports the idea of cooperative standards to combat spam and will
support ‘no
follow’ … in the near future,” he said.

Six Apart, a provider of hosted and licensed blogging software, said it
would turn on
the tag for comments automatically for subscribers to the hosted TypePad
service by the end
of day Wednesday. Users of its MoveableType software can download a
plug-in to add
this functionality.

Six Apart’s Anil Dash, vice president of the professional network, said
TypePad users won’t be able to turn off the no-follow tag. “We try to err on
the side of security. If you’re an advanced user and can edit your HTML, you
can disable it,” Dash said, adding that Six Apart plans to offer more
granular controls in future releases. “Some people want to be able to
share page rank or link credit with the people who leave comments,” he said.

A Google spokesman refused an interview request and referred inquiries to
the company’s blog, which
said that previously posted links on blogs would be “transformed” to include
the no-follow tag.
An FAQ said that individual bloggers “probably” don’t need to do anything.
“Updating the software
that generates these pages will ensure that most bloggers get these changes
automatically,” the FAQ said.

“Google is trying to ensure they continue to improve the user
experience,” said Ellen Siminoff,
CEO of Efficient Frontier, a company that specializes in paid search. “That
will help marketers as
well. We’re thrilled that Google and Yahoo/Overture are continuing to evolve
their natural results,
so the channel continues to grow and be useful for people.”

Sullivan pointed out that the tag is not useful only to bloggers; it can
be used on Web sites as
well. “For the first time, page authors have been given the ability to
control individual elements
of the page,” he said, rather than just telling spiders not to index the entire page.

But it was unclear from the various announcements whether the no-follow
tag could be automatically
implemented across the universe of hosted blogs. That’s an important issue,
Sullivan said, because
most comment spam takes place on abandoned blogs, not active ones. “On blogs
that have been started
and abandoned, you left the door wide open, and now people are ransacking
it,” he said.

Yahoo’s Zawodny said, “This is a technique that requires effort on the
part of the blog owner.
If a blog is abandoned, they probably won’t make a decision like that.
But it may prompt some of
the weblog hosting companies to think more about their policies for hosting
abandoned blogs.”

Said Did-It’s Lee, “If the tag requires an opt-in by the blog owner to
implement it, and abandoned
blogs are the most heavily spammed, it barely makes an impact.”

But Six Apart’s Dash disagreed that abandoned blogs were the major source
of comment spam.
“They do become cesspools and do get attacked, but there’s not a lot of
value in them, and spammers
know this,” he said. “Spammers attack high page-ranked blogs.”

And those authors likely will quickly follow the no-follow rule.

Get the Free Newsletter!

Subscribe to our newsletter.

Subscribe to Daily Tech Insider for top news, trends & analysis

News Around the Web