Microsoft’s Cloud Server Specification Design

Kushagra Vaid General Manager, Cloud Infrastructure Server Engineering Microsoft

2

2.4+ million emails per day

5.8+ billion worldwide queries each month

250+ million

active users

8.6+ trillion objects in Windows Azure

storage

1 in 4 enterprise customers

50+ billion minutes of connections

handled each day

48+ million users in 41

markets

50+ million

active users

200+ Cloud Services 1+ billion customers · 20+ million businesses · 90+ markets worldwide

400+ million

active accounts

Chicago

Cheyenne

Dublin Amsterdam

Hong Kong

Singapore

Japan

San Antonio

Boydton Shanghai

Quincy

Des Moines

Brazil

Australia

Huge infrastructure: scale is the enabler Microsoft has datacenter capacity around the world…and we’re growing

35+ factors in site selection: §  Proximity to customers §  Energy, Fiber Infrastructure §  Skilled workforce

Server scale implications: Design

<10K SMB/Enterprise

100K Hosters

1M Cloud-Scale

# SKUs Several Limited Extremely limited

Redundancy model Hardware based

(Hot-*) Software based

(Local datacenter) Software based

(Geo-distributed)

HW availability 99.999% or higher 99.9% - 99.999% 99% - 99.9%

HW type Enterprise SKU Off-the-shelf design, custom integration

Custom designs, custom integration

Infrastructure co-design

None Limited integration

with Datacenter and Network

OS, Datacenter, Server and Network tightly

integrated

Server scale implications: Supply Chain

<10K SMB/Enterprise

100K Hosters

1M Cloud-Scale

Deployment unit Individual servers 100-1000 servers >10,000 servers

Procurement process

EA / RFP RFP Direct component

sourcing

Materials management

N/A Supplier managed Consignment

Manufacturing N/A Supplier managed Contract

manufacturing

HW quality N/A Supplier managed Self-warranty

Server scale implications: Operations

<10K SMB/Enterprise

100K Hosters

1M Cloud-Scale

Break/fix support 24 hours x 7 days 8 hours x 5 days Up to 1-2 weeks

Issue triage model IT admin Some automation,

admin support Fully automated, machine learning

OOB HW management

Full command set, BMC required

Basic feature set, BMC required

Power On/Off only, No BMC

Management domain scale

100’s of servers 1000’s of servers 10’s of 1000’s

of servers

FRU granularity Hot-swappable

components Component

replacement Entire server replacement

Microsoft cloud server spec: Goals

§  Component re-use

§  Targeted technology refresh

§  Environmental sustainability

Decommission Architecture & Design

§  Commoditized

§  Modular with feature simplicity

§  Power efficient

§  Technology innovation

§ Minimum materials use

§  Low cost mass production

§  Streamlined assembly

§  Industry standard components

Manufacturing & Supply Chain

§  Scalable systems management

§ HW and SW security

§  Reduced operator errors

§ Diagnostics and self-healing

Management & Operations

Microsoft cloud server spec: Infrastructure

▪ Standardization & Modularization

▪ Design Simplicity

▪ Operations Excellence

Key features

Shared infrastructure for efficiency and TCO optimization ▪  Power delivery, mechanicals,

thermals/cooling, management

▪  Optimized for mass contract manufacturing and assembly

▪  Up to 40% cost savings and 15% power efficiency benefits

▪  Saves 10,000 tons of metal per one million installed servers

Compute blade

Signal backplane

Shared power

Shared management

JBOD expansion

Shared fans

12U Shared Chassis EIA Rack Mountable

Blind-mated connectors (12V Power, Ethernet, SAS,

Management)

Signal Backplane

Key features

Blind-mated signal connectivity ▪  Decoupled architecture for server

node and chassis enabling simplified installation and repair

▪  Cable free design for significantly fewer operator errors during servicing

▪  Up to 50% improvement in deployment and servicing time

Key features

Network and storage cabling via backplane architecture ▪  Architectural flexibility for multiple

network types ▪  One-time cable install at factory

▪  No cable touch needed during production operations

▪  Saves 1,100 miles of cable for a deployment of one million servers

2x4 SAS 6G 2x10Gbe SPF+

Key features

Secure & scalable systems management

REST API for chassis infrastructure ▪  TPM boot, SSL transport and role-based

authentication built-in ▪  Simplified server management via extremely

small subset of IPMI commands ▪  Support for server diagnostics and

self-health checks ▪  Up to 75% improvement in operational agility

Management card

Microsoft cloud server spec: OCP contribution Specifications

Chassis, Blade, Chassis Manager, Mezzanines, Management APIs

Mechanical CAD Models Chassis, Blade, Chassis Manager,

Mezzanines

Source Code Chassis management source code through Open Source

/// <summary> /// Gets Fan speed in RPM /// </summary> /// <param name=“fanId">target fan Id</param> /// <returns>Fan speed in RPM</returns> internal FanSpeedResponse GetFanSpeed(byte fanId) {

Board Files & Gerbers Chassis Manager, Tray Backplane,

Power Distribution Backplane

More information: Technical breakouts

Technical Workshop Presenter

Hardware Overview Mark Shaw, Director

Management Software Overview Badriddine Khessib, Director

Blade Overview – Compute & Storage Martin Goldstein, Principal Systems Architect

Chassis Manager Hardware Overview Bryan Kelly, Senior Platform Software Engineer

Visit the Microsoft booth for live demos

Microsoft datacenter resources

Microsoft Datacenters Web Site & Team Blogs www.microsoft.com/datacenters

Windows Azure http://www.windowsazure.com

Office 365 http://www.office365.com

Q & A

Thank you!

© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.