Sunday, July 25, 2010

World Datacenter Storage at 1 ZB

Heard on the BBC World Service:
"The world is drowning in a sea of data. Facebook users alone are uploading more than a thousand photos a second. We're now seeing an exponential explosion of information. So how much information are we really storing?"

There are really two parts to this segment: a backgrounder and an interview. In the first part, the presenter makes the following statements:
  • FB users 1000 pix/sec
  • Digital data stored in the world last year increased by 60%
  • 1 ZB (zettabyte) == 250 billion full movies [Note: 1 ZB == 1000 exabytes]
  • By 2020 world storage will need 35 ZB
The interview is with Matthew Yeager of Computacenter UK LTD, who makes the following claims with regard to data storage and GHG (greenhouse gases) emissions:
  • Cameras, movies, etc. are driving storage requirements
  • YouTube gets uploaded with 24 hrs of vid/min
  • GHG for sending an email attachment of 4.7 MB == 17.5 electric-kettle boils
  • DCs have reached parity with airlines in GHG emissions
  • Google placed its DCs near energy sources, anticipating future gov't tax for energy transport
These claims are nothing if not provocative. Although it's not clear to me how some of the numbers are arrived at, I found similar remarks on Yeager's blog. I also found some additional background info for comparison:
  • Airlines GHG emissions ~ 500 million metric tons/yr
  • According to PBS Roughscience, a standard kitchen kettle takes about 2000 Watts of electrical power. Thus, 17.5 × 2 kW = 35 kW. But that's power (P), not energy (E = P×T) over time period T. If it takes about 10 mins to boil a kettle then, 2 kW / 6 = 0.33 kWh per boil.
  • If a typical British kitchen kettle is assumed to hold 1.5 liters, that's about 6.34 US coffee cups, according to Google convert. According to General Electric, brewing 3 pots of coffee (6 cups) requires 1 kWh of energy. So, 1 brew (or 1 kettle boil) takes about 0.33 kWh, which seems consistent.
  • How much energy does it take to store an email? The claimed 35 kW per email in the BBC piece must combine: client processing + network routing + server processing (e.g., anti-spam) + storage.
Other things not mentioned by Yeager include:
  • Energy consumed by or GHGs due to spam. According to Mcafee, in 2008, spam email alone was responsible for consuming as much as 33 billion kWh—enough to power 2.4 million homes with electricity. Mcafee might be a little bit biased. :)
  • And don't forget spiders
I will continue to contemplate all this, but your comments are also welcome in the meantime.


Chen Shapira said...

I would say much of the discussion is meaningless without discussing trade-offs.

What if those photos uploaded to facebook are all photos not being developed and printed? Does printing photos cause less pollution than storing them in a data center?

What if those non-printed photos are also photos that are not being air-mailed to grandma because grandma watches the grandkids in you-tube and skype? Do we save energy on not sending photos by airmail?

We have less photo albums, so we need less space storing memory in our homes - maybe it saves on air-conditioning energy?

The argument that the more advanced our technology the more energy we use is compelling, but is it true? Does the world as a whole uses more energy every year or do things balance out?

Neil Gunther said...

Richard McDougall tweeted: "Agrees with data I put together recently, media dwarfs legacy"