+ All Categories
Home > Technology > Big Data Behind the Scenes

Big Data Behind the Scenes

Date post: 08-May-2015
Category:
Upload: bret-piatt
View: 830 times
Download: 0 times
Share this document with a friend
Description:
Bret Piatt, General Manager of Apps@Rackspace, will talk about how his businesses use OpenStack, Hadoop, and custom built big data systems to deliver service provider scale and reliability systems. This talk will cover how Rackspace solves real world problems including high level architectures and operational metrics.
17
Delivering email, backup, & collaboration to millions Bret Piatt – General Manager, Apps@Rackspace Email: [email protected] / Twitter: @bpiatt Big Data Behind the Scenes
Transcript
Page 1: Big Data Behind the Scenes

Delivering email, backup, & collaboration to millions

Bret Piatt – General Manager, Apps@Rackspace

Email: [email protected] / Twitter: @bpiatt

Big Data Behind the Scenes

Page 2: Big Data Behind the Scenes

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

• This presentation contains "forward-looking" statements that involve risks, uncertainties and assumptions. If the risks or uncertainties ever materialize or the assumptions prove incorrect, our results may differ materially from those expressed or implied by such forward-looking statements. All statements other than statements of historical fact could be deemed forward-looking, including, but not limited to, any projections of financial information; any statements about historical results that may suggest trends for our business; any statements of the plans, strategies, and objectives of management for future operations; any statements of expectation or belief regarding future events, potential markets or market size; technology developments; and any statements of assumptions underlying any of the items mentioned.

• These statements are based on estimates and information available to us at the time of this presentation and are not guarantees of future performance. Actual results could differ materially from our current expectations as a result of many factors, including but not limited to: the unpredictable nature of our rapidly evolving market and quarterly fluctuations in our business; the effects of competition; and any adverse changes in our indirect channel relationships. These and other risks and uncertainties associated with our business are described in our quarterly and annual reports filed with the Securities and Exchange Commission at www.edgar.gov. We assume no obligation and do not intend to update these forward-looking statements.

• This presentation outlines general information regarding our services and is for informational purposes only; all statements and information are provided “AS IS” and are presented without warranty of any kind, express or implied. Our product/services offerings are subject to change without notice.

• Trademarks and Service Marks• Rackspace® and Fanatical Support® are service marks of Rackspace US, Inc. registered in the United

States and other countries. OpenStack and OpenStack design are trademarks of OpenStack, LLC. Other trademarks and trade names appearing in this presentation are the property of their respective holders. We do not intend our use or display of other companies’ trade names, trademarks, or service marks to imply a relationship with, or endorsement or sponsorship of us by, these other companies.

SAFE HARBORSTATEMENT

2

Page 3: Big Data Behind the Scenes

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Who is Rackspace?

200,000+ CUSTOMERSCUSTOMERS IN 120+ COUNTRIES

5,300+ RACKERSOFFICES IN 8 COUNTRIES

HOSTING PORTFOLIO DEDICATED * CLOUD * HYBRID

LEADER IN GARTNER’S MAGIC QUADRANT

MANAGED HOSTING

OPENSTACK FOUNDERFOUNDATION PLATINUM MEMBER

GLOBAL FOOTPRINTPUBLIC CLOUD IN 6 REGIONS

10 DATA CENTERSWORLDWIDE ON 4 CONTINENTS

***

*

*

*

4 TIME MICROSOFTPARTNER OF THE YEAR

AWARD WINNINGCUSTOMER SERVICE

SHAREPOINT LEADERMOST MVPS, BOOKS, & HOSTING*

*Hosting leadership based on revenue from Microsoft Service Provider License Agreement (SPLA) SharePoint software

Page 4: Big Data Behind the Scenes

4RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Who am I? What is Apps@Rackspace?

• Bret Piatt, General Manager – Apps@Rackspace– 15 years of service provider experience (AT&T and Rackspace)

– Technical operations, architecture, and support background

– Product, corporate development, and corporate strategy experience

• Apps@Rackspace – the SaaS division of Rackspace– 350+ Rackers with major offices in San Antonio, TX; Blacksburg, VA; and Cincinnati, OH

– Business email service with over 3,000,000 paid mailboxes

– SharePoint collaboration service with over 1,000,000 managed seats

– Data backup service with over 10,000,000,000 files protected

Page 5: Big Data Behind the Scenes

5RACKSPACE® HOSTING | WWW.RACKSPACE.COM

What infrastructure runs Apps@Rackspace?

Apps@Rackspace (Email, Collaboration, & Backup)

Page 6: Big Data Behind the Scenes

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Major Trend Keeping Me Up At Night• Storage doubles every 1.7 years

• Network doubles every 2.4 years

• Storage unit costs drop 40% per year

• Network unit costs drop 26% per year

Applications need to be redesigned over time to bring the code to the data or networking costs will become a much larger portion of my infrastructure spend

6

Page 7: Big Data Behind the Scenes

7RACKSPACE® HOSTING | WWW.RACKSPACE.COM

SharePoint@RackspaceCloud computing powering business collaboration

Page 8: Big Data Behind the Scenes

8RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Per User SharePoint – Key Requirements

Development Goals• Rapid time to market

• Easy capacity management

• Scale to 10,000+ customers

• Ability to start small (10 users)

• Ability to grow big (100s of users)

• Efficient infrastructure

Product Offering

• Full versions of SharePoint 2013• Foundation, Standard, & Enterprise

• Active Directory integration

• Fully managed and supported

Page 9: Big Data Behind the Scenes

9RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Per User SharePoint – Capacity On Demand

• Built on OpenStack public cloud infrastructure• Cloud Networks

• Cloud Servers

• Cloud Block Storage

• Cloud Load Balancers

• Cloud Servers Vyatta Appliance (firewall)

• Will use hybrid public/private at scale • RackConnect joining Public & Private

• Private for workload optimized hardware

• Public for capacity on demand

Page 10: Big Data Behind the Scenes

10RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Rackspace EmailThe big data platform behind millions of mailboxes

Page 11: Big Data Behind the Scenes

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Rackspace Email by the Numbers

11

Two billion DNS queries

per day

One billion SQL queries

per day

~150 GB of indexed

data per day

500+ GB of logs per day

Page 12: Big Data Behind the Scenes

12RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Rackspace Email Log Sources for Hadoop

Page 13: Big Data Behind the Scenes

13RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Rackspace Email Hadoop System

Page 14: Big Data Behind the Scenes

14RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Jungle DiskHybrid infrastructure ensuring reliable data backup

Page 15: Big Data Behind the Scenes

15RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Goals for Jungle Disk InfrastructureDevelopment Goals

• Always online

• Fast upload and download

• Cost effective

• Easy capacity management

• Easy to troubleshoot

• Transparent deployments

Product Offering

• Desktop and laptop backup

• Server backup

• Online storage

• Windows, Mac, & Linux

Page 16: Big Data Behind the Scenes

16RACKSPACE® HOSTING | WWW.RACKSPACE.COM

How Jungle Disk Uses Cloud Services

Public cloud for capacity on demand

Dedicated Cloud for

customized fixed

capacity systems

Page 17: Big Data Behind the Scenes

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Bret [email protected]: (210) 312-3302Twitter: @bpiatt

www.rackspace.com

17

17


Recommended