Holistic Approaches to Green IT

Dr Hugh Beedie

CTO

ARCCA & INSRV

Cardiff University

Going Green Can Save Money

Introduction What is Green IT ? Drivers & Metrics Data Centres, Power & Cooling Servers, Virtualisation, and Storage PCs, Monitors & Thin clients Printing and Networks Purchasing policies Influencing user behavior & the organisation ‘Thinking outside the box’ – save energy with

Condor

What is Green IT ?

No environmental impact Get real

Use of resources in an efficient way Better

Avoid ‘Greenwash’ Often faulty comparisons

Importance of your Context Your situation, drivers & metrics… Can change over time

Drivers – Why do Green IT?

Increasing demand for CPU & Storage Lack of Space Lack of Power Increasing energy bills (oil prices doubled) Enhancing the reputation of your University &

attracting better students Sustainable IT & Legal Compliance Because we should (for the Planet )

Metrics – Chicken and Egg

You can’t measure everything.So, which numbers do you

measure, when you don’t know which are important ?

Guesstimation

Which use more, PCs or Servers ?10,000 PCs using

150W, 8 hrs/day, 220 days/yr UK electricity £300,000 p.a.

500 servers using 300W + losses (X 2) 24 X 7 UK prices £300,000 p.a.

So where do we start ?

Examples of Metrics

Cost (Watts, TCO, capital investment, etc)Value (CPU Hrs, MFlops, Gbps, Papers)Value/Cost is better ? (MFlops/Watt)Cooling & power losses (PUE)Average kWhr per PC per yearPages printed (or not printed ?)Cost per page (electricity, paper)Space saved

Power Usage Effectiveness

PUE=Total Power/Computer Power

The DetailWhat wastes the power ?

Power Conversion in HV Transformer Before it gets to your room, you lose power.

Transformer Efficiency = 98% or 95% Return On Investment (ROI)?

Capital Cost = £5500 or £5000 (650kVA) New installation (@200kW), ROI = 1 month Replacement, ROI = 1 year Lifetime of investment = 20+ yrs !!!!!

Data Centre Consumption

Powerdelivery40%

80% 100%

Loads

Cooling

Cum

ulat

ive p

ower Bull View

Source: Intel Corp.

7.3%

Voltage Regulators

20W

5.5%

Server fans 15W

7.3%

UPS +PDU 20W

18.2%

PSU 50W

36.4%Load

CPU, Memory,Drives , I/O

100W

25.5%

Room cooling system

70W

Total 275W

Intel View

Cooling Inside the RoomHot Aisle/Cold Aisle Layout

Hot Aisle Containment

Hot Aisle Ceiling Tiles/Cable Trough

Seals in hot air, prevents mixing with room air Chamber

DoorsAccess to hot aisle, locks for security. Prevents hot air mixing

InRow RC unit

InRow air conditioner cools hot exhaust air directly

Rack Air Containment Airflow Diagram

Server exhaust air is prevented from escaping by rear containment

All exhaust air is returned to InRow cooling unit

Fans of InRow cooling unit distribute cool air to front of IT equipment

Top Down View

Front

Rear

InRow Cooling

Unit

InRow Cooling

Unit

NetShelter SX Rack

Servers

Rear Containment

Front Containment

APC Inline RC units • Up to 20kW/rack• Uniform cooling allows raising cold air temp

Cooling Inside the Room

Temperature (deg C)

<24 27.4 30.8 34.3 >37.8

Cooling – Outside the Room

3 Airedale Chillers

Ultima Compact Free Cool 120

Quiet model

Variable speed fans

N+1 arrangement

(We needed that )

And in real life

Free-cooling

Cooling Load

System operated on 100% Free-Cooling (12% of Year)

System operated on partial Free-Cooling (50% of Year)

System operated on mechanical cooling only(38% of Year)

-7oC3.5oC

12.5oCAmbient Temp.

Mechanical (compressor) coolingFree-cooling

Back Up Power

Full and half load efficiency >92%

Scalable & Modular – could grow as we grew, keeping %max load high

APC 160kW Symmetra UPS

UPS Efficiencies

High Density Servers

Important if you spend on advanced cooling methods

Can have shared power supplies (PSU) Higher efficiency

Blade Servers have shared PSUs Not always good for HPC

Bull supplied half-width, 1U servers. 2, quad core, 80W Harpertown CPUs

Merlin Cluster

Cardiff’s PUE vs Temperature

Average PUE = 1.3

No correlation with temperature ?

Key Power Saving Solutions

Low power servers

High Density Servers (sweating the asset)

Modular UPS

(not over provisioned)

Cooling Infrastructure

Chilled Water“Air Handling”

HVTransformer

98% efficiency

Virtualisation and Storage

Virtualisation Many virtual servers in one physical server

Key questions How much can you save ? How much of that is energy ?

Storage Fast disk=450GB, 25W, Slow disk=2TB, 15W Tiering = Energy saving

Business CaseBefore Virtualisation

Equipment FY Unit Total

100 RM servers - replacement cost, inc installation and peripherals

(assumes Dell PowerEdge 1950)

08/09 £2700 £270K

100 ‘Out of warranty’ servers – 2 year extended warranty 08/09 £500 £50K

250 Production Servers - replacement cost, inc installation and peripherals

(assumes Dell PowerEdge 1950)

09 - 11 £2700 £675K

1 Air conditioning upgrades in RWD 1.71 and Main 1.71 Computer Rooms

08/09 £100K £100K

6 12KVA and 16KVA UPS replacements 09/10 £9K £54K

Total hardware replacement costs (1 for 1) £1.15M

450 Cost of electricity for servers (200w per server @12p/KW hour) Annual £94K £283K

450 Cost of electricity for cooling (100w per server @12p/KW hour) * Annual £47K £142K

Total electricity costs £425K

Total hardware and electricity costs £1.24M

* Inefficient raised floor and ceiling mount air conditioning units

Business CaseAfter Virtualisation

Equipment FY Unit Total

5

RM Servers – replacement cost, inc installation and peripherals

(assumes Dell PowerEdge P900 with 32GB RAM, 2TB Disk)

08/09 £5K £25K

5

‘Out of warranty’ servers– replacement cost, inc installation and peripherals (assumes Dell PowerEdge P900 with 32GB RAM, 2TB Disk)

09/10 £5K £25K

13

Production Servers – replacement cost, inc installation and peripherals

(assumes Dell PowerEdge P900 with 32GB RAM, 2TB Disk)

09/11 £5K £65K

2 24TB Central Storage – assumes Sun Fire X4500 or Equivalent 08/09 £15K £30K

46 VMware ESX Licences – one per CPU 08 - 11 £1K £46K

Physical to Virtual (P2V) Software ? 08/09 ? ?

VMware backup software? 08/09 ? ?

Total hardware costs (VMware Infrastructure) £191K

25 Cost of electricity for servers (500w per server @12p/KW hour) Annual £13K £39K

25 Cost of electricity for cooling (250w per server @12p/KW hour)* Annual £6K £18K

Total electricity costs £57K

Total hardware and electricity costs £248K

* Inefficient raised floor and ceiling mount air conditioning units

Virtualisation Business Case3 Year ROI with Virtualisation

Before Total

Total hardware costs £1.15M

Total electricity costs £425K

Total costs £1.57M

After Total

Total hardware costs (VMware Infrastructure)

£191K

Total electricity costs £57K

Total costs £248K

Hardware £959K

Electricity £368K

Total Savings = £1.32M

Planet Filestore

JISC-funded project to measure savings of..Tiered storage – scenario

100TB needed for 30000 (staff and students) High performance for 7000 concurrent users Site mirrored Raid 10 450GB SAS vs Non mirrored Raid 5 2TB SATA ~ factor of 16 in hardware cost Therefore, use HSM (old data on Tier 2)

What about energy ?

Planet Filestore Project

Fast disk (FC/SAS) 450GB, 25W

Slow disk (SATA) 2TB, 15W

Tiered storageTier 1 – DRTier 2 – non DR

100TB Tier 1 (DR)Energy cost

£25,000 p.a.10TB Tier 1 (DR)90TB Tier 2(nonDR)Energy cost

£3,250 p.a.

Energy saving = £21,750

PCs & Monitors

Where to start ?Duty cycle guesstimate

Always on (100%) On during working

hours (20%)

Turn it off at night !!

What next for PCs ?

More stringent power saving policiesAsset Management (reporting)Influencing user behaviour (Joulemeter)

What about Thin Clients ?

Year PC Thin Client

2005 300W 15W

2010 50W 5W

Still some scope, but notwhat there used to be

Printing

Thanks to Liverpool John Moores University (LJMU) for the next 2 slides

Carbon Footprint Savings

• Hundreds of devices removed– Old lasers– Deskjets– Fax machines

• Replaced with fleet of 220 devices

• Automatic power saving after 2 hours

• Use of recycled paper

• FollowMe documents held for 18 hours

Actual Savings at LJMU

Networks

Power used by switches 3.5Gbps/W to 0.5Gbps/W

Cardiff LAN has 50,000 outlets 100kW down to 15kW Potential £85k p.a. by choosing well

PoE (up to 25W down a UTP cable) 5W in the cable, 5W DC-DC conversion 40% loss

Use EnergyStar PCsDiscourage use of the fastest PCsUse energy in your TCO calculationsEnsure suppliers only publicise green PCs

Purchasing Policies

Influencing Behaviour

Context – who pays for electricityTarget your messages

Users• Their PC uses ???W• Pages printed last week ??

Technical types – detail• MFlops, kW, Geeks/Pint

Managers• Bottom line ££££• Reputation.

Finally….

What is Green, but makes your PC use more electricity ?

Answer – CondorSo, what is Condor ?Condor takes spare CPU Cycles for use in

Research calculations Runs at low priority so users don’t notice

So why is it Green ?

Condor - 2.4GHz Core2Duo

0

50

100

Idle (W) 64.5

Max (W) 80.7

Conclusions

Measure, Improve, Measure againLook everywhere for wasteContext changes over time

Make sure you revisit your assumptions

Don’t just do the technical stuffThink out of the box (Condor)

Questions ?

Cardiff’s New Datacentre

£3M grant to purchase a supercomputerNew Datacentre required with appropriate

power, cooling ..Our Context ?

INSRV Sustainability Mission - To minimise CU’s IT environmental impact and to be a leader in delivering sustainable information services.

Limited experience in Datacentre build Server & Data Center Energy Efficiency

(US Congress Report 2007)

What did Cardiff do ?

Ran 2 tenders, many contractors (& subs) HPC & Environment (power, cooling)

Strong project management TCO as key evaluation criterion Value = HPC Benchmark figures Winner = Highest Value/TCO