So you have an idea for a product or service. How do you know it’s viable? Simple. Do a proof of concept.
For decades, companies have used proof of concept (PoC) projects to test new technologies and attract investors and encourage partners.
But as PoC results begin moving from research reports to marketing collateral, industry watchers are becoming increasingly skeptical about their usefulness.
Adding to his jaundiced view is the fact that some companies are spending millions of dollar to fund demonstrations, which seem bent pushing their own brands.
So if PoCs are being abused, how will customers know what to believe?
Gary Allbee is president of Alta Industrial Automation, a design services firm that helps other companies bring their PoC designs to life. The Calgary, Canada-based company specializes in 32-bit embedded systems but is also branching out into custom Palm-based computing designs and wireless Internet access. In his 13-year career, he’s seen some suspect designs cross his desk.
“I have seen proof of concepts that are just a ploy to get investors,” Allbee told internetnews.com. “The investors come in a lot faster at the proof of concept stage because you are eliminating the technology risk. You are telling them in essence that you’ve gone 10 feet and you need to go 20 feet, but the extra $10 million or $100 million would be helpful.”
Allbee tries to counter outrageous PoC proposals. “There is no anti-gravity device that can get you to Mars,” he said jokingly.
This is not to say that all companies are perpetrating PoC scams. Proof of concepts are regularly used by IBM, , Microsoft
, HP
, Intel
and Sun Microsystems
. Each has used the tests to improve products and gain customer feedback.
But the idea that companies are using a scientific process in a quid pro quo for marketing ink troubles some. To understand how the process is to look at Intel’s production of its Itanium processor.
In the late 1990s, Santa Clara, Calif.-based Intel worked with the Global 2,000 on testing the EPIC-architecture server chip in the field. Rhett Livingood, director of market and business development for Intel Solution Services, said the chipmaking giant was approached by several customers including Bell Canada, CBS Broadcasting, CFE, Chevron/Texaco, CitiStreet, First Trust, and British Petroleum.
“With BP, they were looking for a replacement for their seismic division,” Livingood said. “They had a big SGI installation and were looking at Itanium so they could do even more production. We set up a pilot for them running on their existing infrastructure. By and large, we find some areas of improvement. After that, we introduce new hardware or new software, such as some new realms of a multi-processor or dual-processors and we do some comparison in sequence.”
The tests were successful and the customer bought a few systems based on Itanium, but Intel and BP often talk about the experience in marketing materials and other presentations. Livingood said the BP agreement includes joint marketing collateral and it validates the technology but that many of its partners know well in advance what Intel is developing.
“These projects last six to nine months — from business problem to pilot phase and these are very long-term projects that can take from two to three years,” Livingood said. “Some of the testing is confidential. It depends on the customer’s preferences. In general, however our partners are comfortable with talking about their testing with us openly.”
Illuminata senior analyst and IT advisor Gordon Haff said the practice is more widespread than people realize.
“Certainly when you are trying to get a technology up and running, you need to get momentum,” Haff told internetnews.com. “Early customers and big marquee names get some pretty good deals in exchange for some credence that they lend in a new architecture. Even the savviest customer is going to have problems approaching a CIO with a new concept when the executive is more concerned about the bottom line.”
Haff said that is one of the reasons you hear the same names mentioned over and over in marketing materials and in corporate presentations.
“The earliest gear is given for free,” he said. “Often there are beta systems that a customer does not pay for. But in general, companies are very interested in the technology and lying about the test experience would be a rare and extreme case.”
“More likely is that they get early access to equipment or a good discount for helping vendor out,” Haff continued. “Out and out companies are saying ‘here is a bunch of money to appear in a press release.'”
Some even pay for the PoC itself. Sun runs between 60 and 70 Competency Centers around the world that are funded in part by the company (as much as $5 million in some cases) as well as its iForce partners and other investors.
A.J. Mahajan, Sun group manager of enterprise system products, said the company and its partners use the centers and proof of concept programs to pre-test reference architectures so that customers can save time and money.
“When Sun came out with Solaris 9 and SPARC III and Oracle came out with its 9i applications, customers asked us if they had to change their entire infrastructure,” Mahajan told internetnews.com “We pre-tested the systems and signed off on the documents. The customers see that if these things work, they can test these for availability on their own systems. It would have taken them six months to a year to do it by themselves and cost a lot of money.”
Mahajan said the difference for Sun is that because it pays for the centers in advance with its partners, it does not charge for testing. The company is also vendor neutral and customers are not forced to buy discounted hardware or software.
“They literally bundle these settings and send them over to their data centers,” he said. “They get a running start and they don’t have to manually reconstruct things there.”
Allbee said customers must do their homework and keep in mind the true purpose of the PoC model.
“There is a concept for going faster than the speed of light so hurry and patent it,” Allbee said. “But you find out later that it defies the laws of our science and is not possible and a lot of time and money was wasted over the initial excitement.”
