Photos of Larryblakeley
(Contact Info: larry at larryblakeley dot com)
Important Note: You will need to click this icon to download the free needed to view most of the images on this Web site - just a couple of clicks and you're "good to go."
Computing economics are changing. Today there is rough price parity between (1) one database access, (2) ten bytes of network traffic, (3) 100,000 instructions, (4) 10 bytes of disk storage, and (5) a megabyte of disk bandwidth. This has implications for how one structures Internet-scale distributed computing: one puts computing as close to the data as possible in order to avoid expensive network traffic.
Hardware and software are minor parts of the total cost of ownership. Hardware comprises less than half the total cost; some claim less than 10% of the cost of a computing.
Outsourcing works when it is a service business where computing is central to operating an application and supporting the customer – a high-tech low-touch business. It is difficult to achieve economies-of-scale unless the application is nearly identical across most companies – like payroll or email. Some companies, notably IBM, Salesforce.com, Oracle.com and others are touting outsourcing, "On Demand Computing," as an innovative way to reduce costs. There are some successes, but many more failures. So far there are few outsourced megaservices – payroll and eMail are the exception rather than the rule.
Neither grid computing nor web services have an outsourcing or advertising business model. Both are plumbing that enable companies to build applications. Both are designed for computer-to-computer interactions and so have no advertising model – because there are no eyeballs involved in the interactions. It is up the companies to invent business models that make this plumbing useful.
Grid computing and computing-on-demand enable applications that are mobile and that can be provisioned on demand. What tasks are mobile and can be dynamically provisioned? Any purely computation task is mobile if it is written in a portable language and uses only portable interfaces -- write once run anywhere (WORA). Cobol and Java promises WORA. Cobol and Java users can attest that WORA is difficult to achieve, but for the purposes of this discussion, let's assume that it is a solved problem. Then, the question is:
What are the economic issues of moving a task from one computer to another or from one place to another?
A task has four characteristic demands:
- Networking – delivering questions and answers,
- Computation – transfor ming information to produce new information,
- Database Access – access to reference information needed by the computation,
- Database Storage – long term storage of information (needed for later access).
The ratios among these quantities and their relative costs are pivotal. It is fine to send a GB over the network if it saves years of computation – but it is not economic to send a kilobyte question if the answer could be computed locally in a second.
The ideal mobile task is stateless (no database or database access), has a tiny network input and output, and has huge computational demand. For example, a cryptographic search problem: given the encrypted text, the clear text, and a key search range. This kind of problem has a few kilobytes input and output, is stateless, and can compute for days.
Most web and data processing applications are network or state intensive and are not economically viable as mobile applications. An FTP server, an HTML web server, a mail server, and Online Transaction Processing (OLTP) server represent a spectrum of services with increasing database state and data access. A 100MB FTP task costs 10 cents and that cost is 99% network cost. An HTML web access costs 10 microdollars and is 88% network cost. None of these applications fits the cpu-intensive stateless requirement.
Data loading and data scanning are cpu-intensive; but they are also data intensive , and therefore not economically viable as mobile applications. The break-even point is 10,000 instructions per byte of network traffic or about a minute of computation per MB of network traffic.
Few computations exceed that threshold; most are better matched to a Beowulf cluster. In a Beowulf cluster networking is ten thousand times less expensive – which makes it seem nearly free by comparison to WAN networking costs.
Still, there are some computationally intensive jobs that can use Grid computing . Render-farms for making animated movies seem to be a good candidate for Grid computing. Rendering a frame can take many cpu hours, so a Grid-scale render farm begins to make sense. For example, Pixar's Toy Story 2 images are very cpu intensive – a 200 MB image can take several cpu hours to render. The instruction density was 200k to 600k instructions per byte . This could be structured as a grid computation – sending a 50MB task to a server that computes for ten hours and returns a 200MB image.
Put the computation near the data . The recurrent theme of this analysis is that "On Demand" computing is only economical for very cpu-intensive (100,000 instructions per byte or a cpu-day-per gigabyte of network traffic) applications.
How do you combine data from multiple sites? Many applications need to integrate data from multiple sites into a combined answer. The arguments above suggest that one should push as much of the processing to the data sources as possible in order to filter the data early (database query optimizers call this "pushing predicates down the query tree"). There are many techniques for doing this, but fundamentally it dovetails with the notion that each data source is a web service with a high-level object - oriented interface.
Beowulf clusters have completely different networking economics. Render farms, materials simulation, and CFD fit beautifully on Beowulf clusters because there the cost of networking is very inexpensive: a GBps Ethernet fabric costs about 200$/port and delivers 50MBps, so Beowulf networking costs are comparable to disk bandwidth costs – 10,000 times less than the price of Internet transports. That is why rendering farms and BLAST search engines are routinely built using Beowulf clusters. Beowulf clusters should not be confused with Internet-scale Grid computations.
If telecom prices drop faster than Moore's law, the analysis fails. If telecom prices drop slower than Moore's law, the analysis becomes stronger. Most of the argument in this paper pivots on the relatively high price of telecommunications. Over the last 40 years telecom prices have fallen much more slowly than any other information technology. If this situation changed, it could completely alter the arguments here. But there is no obvious sign of that occurring.
- "Distributed Computing Economics," Jim Gray http://www.larryblakeley.com/experts/jim_gray/jim_gray.html, Microsoft Research, Microsoft Corporation, Technical Report MSR-TR-2003-24, March 2003 http://research.microsoft.com/research/pubs/view.aspx?tr_id=655
File Name: jim_gray200303.pdf
Post Date: March 23, 2005 at 8:55 AM CST; 1455 GMT