Look out, Google! Jimmy Wales is coming for you. The founder of Wikipedia is aggressively building out his Wikia effort, a commercial collection of Web sites anyone can edit. One of its key parts? Search.
Today Wikia announced the acquisition of the Grub Wales hired Jabber founder Jeremy Miller to lead the Wikia search project, which launched in December. One of the first things they did, Wales said, was look for technologies to help jump start the process.
distributed search project from LookSmart and released the technology under the GPL version 2 open source license. Financial terms were not disclosed.
Much like how Wikipedia and Wikia open the process of content
contribution to the masses, Wikia Search will open up search to the masses.
“The interesting thing about Grub is that it’s a distributed crawler,” Wales
explained. “So people download the client and the client does the crawl
under the direction of a central server. It’s very robust; they [Looksmart]
used it in production for several years.”
Though Wikia now has Grub, Wales was quick to note that it doesn’t preclude
it from using other tools, such as Nutch, which is an open source search
technology built on top of Apache Lucene.
The Grub client is currently available for people to download and test.
Beyond that Wales noted that it’s too early to talk about how it will roll
out.
“If we get a fair number of people using the client and are able to do a
reasonable number of crawls by the end of the year, I’ll be satisfied,”
Wales said.
Ultimately though the larger goal for Wales and his Wikia Search is nothing
short of challenging the search giants themselves.
“Basically we’re trying to build the complete search engine infrastructure,
the whole package and Grub is one of the pieces of the puzzle,” Wales said
confidently. “Once we’re done we do anticipate competing directly with
Google and Yahoo.”
