Declaring that Internet search is currently broken, Wikipedia co-founder Jimmy Wales is trying to fix it. And he wants your help.
After several months of development and acquisitions to assemble the infrastructure needed to build a viable search engine, Wikia Search service goes live today. Wikia Search is a product of Wikia Inc., a for-profit company wholly separate from the Wikimedia Foundation, the nonprofit organization that was founded in 2004 and oversees Wikipedia.
Following a brief period of private alpha testing, today’s unveiling is billed as a “public alpha release.” Wales has freely admitted that today’s launch is something of a test drive, and that the application will continue to see substantial modifications and updates.
With the community-driven, open-source search engine, Wales is hoping to transform Internet search, which he describes as currently dominated by a few large companies operating under a cloak of secrecy with virtually no accountability for how results are served up or pages are ranked.
“The philosophical background here is that I’m a very big advocate of openness and transparency,” Wales told InternetNews.com. The best parts of the Web are largely absent from the proprietary protocols of the major search engines, he said.
“My view here is that there’s an opportunity to change that.”
Starting from www.wikia.com, users can navigate to an unadorned search page that Wales compared to Google’s for its deceptive spareness — like Google, the Wikia Search launch page is just the tip of the iceberg.
Wales said that Wikia Search is both a social network and a search engine. On the site, users register, create profiles and are able to perform all the typical functions of a social networking site – messaging friends, uploading photos, etc.
The search engine serves up results as a series of links ranked in order of relevance, just as Google and Yahoo do (enter a query; receive a list of results), but the similarities end there.
Applying the collaborative editing approach that is the essence of a wiki, registered users can contribute to a mini-article that appears at the top of the results page, Wales said.
Users can click on the links and rank the relevance of the result to the search term on a scale of one to five; their evaluations are then fed back into the algorithm to improve the quality of information delivered.
Building a search engine from the ground up, with results delivered and refined the using the same approach that made Wikipedia the Web’s favorite general reference site, is not going to be an overnight process, Wales admitted. Just looking at the near future, “we’d be ecstatic if we grabbed 5 percent of the market,” Wales said.
The push toward community-driven search began in earnest in July, when Wikia announced the acquisition of Grub, the open source search project that harnesses the distributed processing power of individuals’ computers to crawl the Web.
The Grub Web crawler is just one of many technologies cobbled together to power Wikia Search, Wales said. On the back end, Apache efforts Lucene and Nutch shoulder the brunt of the search and indexing responsibilities. The front end, which handles the user interactions, such as the comments and rankings, is written in Java.
Wales said that Wikia Inc. is still exploring monetization methods for the search engine, but that, in a general sense, it would employ some form of an advertising model. No commercial partners have been announced.
Delivering search results through open source algorithms that evolve through community rankings begs the same question of user abuse that has dogged Wikipedia from time to time.
Search engine optimization is a microindustry in its own right these days. Asked about the concern that by turning the algorithms and rankings over to the community, Wikia Search could be a ripe target for gaming the search engine to artificially boost site rankings, Wales demurred.
“My view is that sunlight is the best disinfectant,” he said. “Talk with security experts and they will tell you that security through obscurity is a bad idea,” adding that the policing of Wikia Search will work in a very similar manner as it does on Wikipedia.
“We have a community that is in charge of the editorial content” that will be able to identify and block the bad apples who aren’t playing by the rules. Ultimately, SEO will be less of an issue for Wikia Search than it is currently with Google, Yahoo and the other major search engines, Wales said.
“I feel that this is going to be one of our strengths.”
Wales said he’s taking an organic approach to the development of the editorial community and that his own involvement will be extensive.
He wasn’t didn’t specifically answer whether there’ll be censorship or banning of rogue community members for trying to game the system?
“I have a general philosophy that’s served me very well: Avoid excessive a priori thinking – wait and see … we’ll see how it unfolds.”
