Enterprise Search And Destroy

Reporter's Notebook: New government regulations often spawn whole new markets. A far-reaching reform of the Federal Rules of Civil Procedure (FRCP) is proving to be no exception.

The reform means that electronic documents in all forms, including e-mail, instant messages and even transcripts of video conference and VoIP  calls, are fair game for litigants during the discovery phase of a lawsuit. This has given enterprise search application vendors significant new momentum.

However, allowing customers to find relevant documents among billions of pages is no mean feat: The CEO of one vendor said he told a prospective customer that it would take between 200 and 300 servers (in addition to the ones already in place) to index some 300 terabytes of data. He was informed that his competition had quoted somewhere in the range of 1,000 servers.

According to IDC analyst Sue Feldman, the field of electronic document search and retrieval has "gone from practically zilch three years ago" to achieving 30 percent growth over the past year, in large part thanks to the changes to the FRCP.

"Text mining is growing at an enormous rate," she told me.

No wonder that established enterprise search application vendors like Autonomy, Recommind and Exalead have introduced new products or tweaked existing software to meet demand for the efficient storage and retrieval of electronic communications.

The reform has also given new life to specialized vendors, such as Toronto-based Nstein, Palo Alto, Calif.-based Attensity, Attenex, based in Seattle, and Nexidia, based in Atlanta.

The reform, which took effect on Dec. 1, has also spawned a rash of new partnerships and acquisitions between traditional enterprise storage companies, such as IBM , EDS , EMC  and CA nbsp;on the one hand, and the document retrieval specialists on the other.

For instance, Exalead CEO Alain Heurtebise told me that his Paris-based company closed a deal with EDS's Italian subsidiary in 2006, and is being considered by IBM as an OEM partner for a storage and e-discovery application.

E-mail storage vendor Zantaz also recently bolstered its feature set with the acquisition of data-classification vendor Singlecast.

Paris-based business intelligence vendor Business Objects  picked Attensity to be its search partner in November, while Nstein has recently signed deals with Cognos  and Computer Sciences Corp. .

E-discovery application vendors promote their own special sauce for allowing corporate lawyers to sift through reams of data while de-duping and otherwise reducing irrelevant search results.

The challenge with searches in this context is that it isn't easy to know what you're looking for. For instance, legal beagles wouldn't have known to search for terms such as "manipulate the California energy market" at Enron without the benefit of hindsight.

According to Heurtebise, Exalead addresses this problem through what he called "serendipitous search," which allows customers to refine or redirect their searches based on an initial set of results. "The initial result sets could give you intelligence and insight about the right question to put. If you're ignorant about what you're looking for, you're obliged to go by serendipity," he said.

Next page: Digging into the words