In a move to improve online ad campaign measurement, auditing specialist ABCi
Monday pledged to create and maintain a master list of spiders and robots, which are widely acknowledged as the main culprit for
inaccurate measurements, for the Interactive Advertising Bureau (IAB).
Why is a list important? Spiders and robots automatically search Web content for information and are used to: retrieve data about
content for search engines, gather content pages for personal offline viewing and track advertising activity by Web site and by
advertiser and identification of “What’s new information”. Because there may be multiple robots or spiders on a site, sites may have
a problem distinguishing between non-human spider and robot activity and consumer interactions, which throws off the metrics
reported to advertisers.
Robin Webster, president and chief executive officer of the IAB, extolled the virtues his organization’s plan, claiming the
robot/spider list will cut back “inconsistent counts.”
“The primary benefits of the list will be to reduce the time, effort and cost to publishers, third party ad servers and agencies
associated with report reconciliation, and to provide advertisers with accurate measurement metrics,” Webster said. “By utilizing
this approved and constantly updated master list, publishers will be able to filter out robot traffic from their reporting to
advertisers, instilling greater confidence in the medium.”
The IAB is in the midst of a multi-step process to hash out ad measurement protocol. The firm established a media glossary last week when it was
determined that third-party ad servers and publishers defined such terms as ad impressions, unique users, clicks, total visits and
page impressions, disparately.
Once the robot/spider list is in place, IAB said the next step will be for ABCi to examine the effect of spiders and robots on the
two different methods of ad serving: client-initiated and server-initiated, followed by the release of measurement guidelines by
year-end.
Having maintained a robot list compile from over 8,000 audits and 200 trillion transactions since May 1995, it is logical that ABCi
is qualified to compile the list and conduct further examinations of the ad measurement process. ABCi invites other organizations to
suggest updates to the list, so long as proper documentation is supplied. On the other hand, ABCi also welcomes challenges to the
list, also accompanied by proper documentation.
Available for free to IAB members and ABCi clients, the robot/spider list will be updated monthly, and serves as the third step in a
slew of recommendations the IAB will post by year’s end.
