Wednesday, October 28, 2009

Googling Google + Linux

Honorary Guerrilla alumnus, GB, sent me a link to "How Google uses Linux" which, although it provides an interesting view inside The Goog's datacenter management, looks like it's supposed to be available only to LWN subscribers:
"The following subscription-only content has been made available to you by an LWN subscriber." Eh? 
Not wishing to let any cats out of their subscription bag, I checked with the editor and he said it's ok to blog the link.

So, with permission, some highlights :
  • Google's way of doing things is far from ideal.
  • As much as 3/4 of Google's code consists of changes to the core Linux kernel.
  • In the area of CPU scheduling, Google found the move to the completely fair scheduler (CFS) to be painful.
  • Work is segmented into three classes: "latency sensitive," which gets short-term resource guarantees, "production batch" which has guarantees over longer periods, and "best effort" which gets no guarantees at all.
  • A lot of Google's code is there for (performance) monitoring.
Reader comments are also worth reading. I'll be comparing CFS with TSS (generic Unix time share) and FSS (fair share) in my upcoming Guerrilla Capacity Planning class.

3 comments:

Stefan Parvu said...

A lot of PCs running their business and I
bet no easy life in keeping all these
running together and measuring them.

Good point they have a performance monitoring in place, but wonder how are they analysing and managing the power
consumption used by these PCs ? Probable
they are looking into:

- consolidation
- workload management

GCaP is one way to attack all these issues.

Stefan Parvu said...

A lot of PCs running their business and I
bet no easy life in keeping all these
running together and measuring them.

Good point they have a performance monitoring in place, but wonder how are they analysing and managing the power
consumption used by these PCs ? Probable
they are looking into:

- consolidation
- workload management

GCaP is one way to attack all these issues.

Neil Gunther said...

Those tens of thousands of "PCs" are actually blades.