Photos of Larryblakeley

(Contact Info: larry at larryblakeley dot com)

Important Note: You will need to click this icon to download the free needed to view most of the images on this Web site - just a couple of clicks and you're "good to go."

Combining more than 15,000 commodity-class PCs with fault-tolerant software creates a solution that is more cost-effective than a comparable system built out of a smaller number of high-end servers.

Here we present the architecture of the Google cluster, and discuss the most important factors that influence its design: energy efficiency and price-performance ratio. Energy efficiency is key at our scale of operation, as power consumption and cooling issues become significant operational factors, taxing the limits of available data center power densities.

Our application affords easy parallelization: Different queries can run on different processors, and the overall index is partitioned so that a single query can use multiple processors. Consequently, peak processor performance is less important than its price/ performance. As such, Google is an example of a throughput-oriented workload, and should benefit from processor architectures that offer more on-chip parallelism, such as simultaneous multithreading or on-chip multiprocessors.

Google architecture overview

Google's software architecture arises from two basic insights. First, we provide reliability in software rather than in server-class hardware, so we can use commodity PCs to build a high-end computing cluster at a low-end price. Second, we tailor the design for best aggregate request throughput, not peak server response time, since we can manage response times by parallelizing individual requests.

We believe that the best price/performance tradeoff for our applications comes from fashioning a reliable computing infrastructure from clusters of unreliable commodity PCs.

Serving a Google query

To provide sufficient capacity to handle query traffic, our service consists of multiple clusters distributed worldwide. Each cluster has around a few thousand machines, and the geographically distributed setup protects us against catastrophic data center failures (like those arising from earthquakes and large-scale power failures). A DNS-based load-balancing system selects a cluster by accounting for the user's geographic proximity to each physical cluster. The load-balancing system minimizes round-trip time for the user's request, while also considering the available capacity at the various clusters.

A hardware-based load balancer in each cluster monitors the available set of Google Web servers (GWSs) and performs local load balancing of requests across a set of them.

In summary, Google clusters follow three key design principles:

- Software reliability. We eschew fault-tolerant hardware features such as redundant power supplies, a redundant array of inexpensive disks (RAID), and highquality components, instead focusing on tolerating failures in software.

- Use replication for better request throughput and availability. Because machines are inherently unreliable, we replicate each of our internal services across many machines. Because we already replicate services across multiple machines to obtain sufficient capacity, this type of fault tolerance almost comes for free.

- Price/performance beats peak performance. We purchase the CPU generation that currently gives the best performance per unit price, not the CPUs that give the best absolute performance.

- Using commodity PCs reduces the cost of computation. As a result, we can afford to use more computational resources per query, employ more expensive techniques in our ranking algorithm, or search a larger index of documents.

- "Web Search for a Planet: The Google Cluster Architecture," Luiz Andre Barroso, Jeffrey Dean http://labs.google.com/people/jeff/, Urs Holzle, Google Labs Research Papers http://labs.google.com/papers/index.html, IEEE Computer Society http://www.computer.org/, IEEE Micro http://www.computer.org/micro/, Vol. 23, No. 2, pages 22-28, March, 2003 http://www.computer.org/micro/mi2003/m2022.pdf

Directory: http://www.larryblakeley.com/Articles/clusters_grids/

File Name: barroso_dean_holze200304.pdf

Post Date: June 4, 2005 at 10:10 AM CDT; 1510 GMT