Scale grid computing down to size

Grid computing captured imaginations by creating ‘Net-based virtual supercomputers out of hundreds of thousands of existing computers. But like many big ideas, grid computing is fraught with challenges, and the best opportunities might prove quite different from what pioneers envisioned.

Grid computing captured imaginations by creating ‘Net-based virtual supercomputers out of hundreds of thousands of existing computers. But like many big ideas, grid computing is fraught with challenges, and the best opportunities might prove quite different from what pioneers envisioned.

The technical benefits of aggregating the computational power of computers residing on the same network are clear. But the business benefits must be weighed against concerns such as cost, security and manageability.

There are three radically different types of grid computing: ‘Net-based, enterprise and supply-chain.

From the outset, visionaries targeted the most distant of the three: leveraging the Internet as a source of unprecedented, on-demand computational power. For example, Platform Computing‘s distributed computing software was used to link 70,000 computers via the ‘Net to map, classify and compare more than 500,000 proteins (the Decrypthon Project).

There are two obvious drawbacks to ‘Net-based grid computing: It is voluntary and vulnerable to the usual Internet plagues. Perhaps someday users will be able to rent processor time over the ‘Net, but first someone must figure out how to compensate the sellers and indemnify the buyers.

Fortunately, there is a more practical and urgent need for grid computing: satisfying the computational requirements of industries such as life sciences research, financial services, and oil and gas exploration.

Still, there are competing visions of grid computing. Grid computing vendors point out that most of a company’s computational power is wasted. But is that a sufficiently compelling reason to implement grid computing? Companies that search for oil or new drugs accept some inefficiency and waste as part of the cost of doing business.

Companies that rely heavily on computational workflow are the best candidates for grid computing, although even in these cases it is more a matter of better systems integration than recovering every last processor cycle. The need to integrate server farms across select departments is far greater than the need to optimize enterprisewide PC utilization.

Supply-chain grid computing is a giant step above enterprise grid computing and a small step below ‘Net-based grid computing. Supply-chain grid computing will facilitate collaboration between business partners, but it requires security and management tools that don’t exist. Ensuring security and performance when you have end-to-end control is hard enough; doing so across corporations is nearly impossible.

With big ideas, customers are expected to take leaps of faith. Grid computing will succeed by letting specific industries create grids spanning specific departments.

Rather than tying together all the computers in a company at once, they will accomplish the same thing gradually. Eventually, “grid computing” and “networking” will be synonymous.